encoderPro/UNRAID-DEPLOYMENT.md

encoderPro - Complete Deployment Guide

THE OFFICIAL SETUP GUIDE - START HERE

This is the complete deployment guide for encoderPro. It covers installation on Unraid with support for NVIDIA GPUs, Intel Arc GPUs, or CPU-only encoding.

Note: This guide uses direct docker run commands for Unraid deployment. The docker-compose.yml file is optional and not required for Unraid.

---

Quick Start for Unraid

This guide shows you how to deploy encoderPro on Unraid with support for NVIDIA GPUs, Intel Arc GPUs, or CPU-only encoding.

Choose Your Hardware

Option 1: NVIDIA GPU (NVENC)

• Best for: RTX 2000-4000 series owners
• Speed: Very fast (100-300 fps)
• Codecs: H.264, H.265
• Follow: NVIDIA Setup

Option 2: Intel Arc GPU (QSV)

• Best for: Arc A-Series or Intel integrated graphics users
• Speed: Very fast (80-250 fps)
• Codecs: H.264, H.265, AV1 (Arc only!)
• Follow: Intel Arc Setup

Option 3: CPU Only

• Best for: No GPU or testing
• Speed: Slow (2-10 fps)
• Codecs: All software codecs
• Follow: CPU Setup

---

NVIDIA GPU Setup

Prerequisites

1. Install NVIDIA Plugin:

   • Go to Unraid Plugins
   • Install "Nvidia-Driver" plugin from Community Applications
   • Reboot

2. Verify GPU:

nvidia-smi

Installation

1. Create app directory:

mkdir -p /mnt/user/appdata/encoderpro
cd /mnt/user/appdata/encoderpro

2. Copy project files:

   • Place all encoderPro files in /mnt/user/appdata/encoderpro

3. Build Docker image:

docker build -t encoderpro:latest .

4. Fix directory permissions:

# Create and set ownership (UID 1000)
mkdir -p /mnt/user/appdata/encoderpro/{db,logs}
mkdir -p /mnt/user/temp/encoderpro-work
mkdir -p /mnt/user/archive/movies
chown -R 1000:1000 /mnt/user/appdata/encoderpro/{db,logs}
chown -R 1000:1000 /mnt/user/temp/encoderpro-work
chown -R 1000:1000 /mnt/user/archive/movies

5. Copy config:

cp config-nvidia.yaml config.yaml
nano config.yaml  # Adjust paths if needed

6. Start Container:

docker run -d \
  --name encoderpro-dashboard \
  --runtime=nvidia \
  -e NVIDIA_VISIBLE_DEVICES=all \
  -e NVIDIA_DRIVER_CAPABILITIES=compute,video,utility \
  -e DASHBOARD_HOST=0.0.0.0 \
  -e DASHBOARD_PORT=5000 \
  -p 5000:5000 \
  -v /mnt/user/movies:/movies \
  -v /mnt/user/archive/movies:/archive \
  -v /mnt/user/temp/encoderpro-work:/work \
  -v /mnt/user/appdata/encoderpro/config.yaml:/config/config.yaml:ro \
  -v /mnt/user/appdata/encoderpro/db:/db \
  -v /mnt/user/appdata/encoderpro/logs:/logs \
  --restart unless-stopped \
  encoderpro:latest dashboard

7. Access Dashboard:
   • Open: http://your-unraid-ip:5000

Unraid Template (NVIDIA)

Add Container Settings:

• Name: encoderpro-dashboard
• Repository: encoderpro:latest
• Network Type: bridge
• Extra Parameters: --runtime=nvidia

Environment Variables:

NVIDIA_VISIBLE_DEVICES=all
NVIDIA_DRIVER_CAPABILITIES=compute,video,utility
DASHBOARD_PORT=5000

Path Mappings:

/movies           → /mnt/user/movies (read-only)
/archive          → /mnt/user/archive/movies (read-write)
/work             → /mnt/user/temp/encoderpro-work (read-write)
/config/config.yaml → /mnt/user/appdata/encoderpro/config.yaml (read-only)
/db               → /mnt/user/appdata/encoderpro/db (read-write)
/logs             → /mnt/user/appdata/encoderpro/logs (read-write)

Port:

5000:5000

---

Intel Arc Setup

Prerequisites

1. Verify GPU detected:

lspci | grep -i vga
ls -la /dev/dri/  # Should show renderD128 and card0

2. No special Unraid plugins needed - drivers install in container

Installation

1. Create app directory:

mkdir -p /mnt/user/appdata/encoderpro
cd /mnt/user/appdata/encoderpro

2. Copy project files:

   • Place all encoderPro files in /mnt/user/appdata/encoderpro

3. Build Intel Docker image:

docker build -f Dockerfile.intel -t encoderpro-intel:latest .

(This takes 5-10 minutes - installs Intel drivers)

4. Fix directory permissions (CRITICAL!):

# Create directories
mkdir -p /mnt/user/appdata/encoderpro/db
mkdir -p /mnt/user/appdata/encoderpro/logs
mkdir -p /mnt/user/temp/encoderpro-work
mkdir -p /mnt/user/archive/movies

# Set ownership to UID 1000 (container's encoder user)
chown -R 1000:1000 /mnt/user/appdata/encoderpro/db
chown -R 1000:1000 /mnt/user/appdata/encoderpro/logs
chown -R 1000:1000 /mnt/user/temp/encoderpro-work
chown -R 1000:1000 /mnt/user/archive/movies

5. Copy Intel config:

cp config-intel.yaml config.yaml
nano config.yaml  # Adjust settings for your Arc model

6. Start Container:

docker run -d \
  --name encoderpro-dashboard-intel \
  --device=/dev/dri:/dev/dri \
  -e GPU_TYPE=intel \
  -e DASHBOARD_HOST=0.0.0.0 \
  -e DASHBOARD_PORT=5000 \
  -p 5000:5000 \
  -v /mnt/user/movies:/movies \
  -v /mnt/user/archive/movies:/archive \
  -v /mnt/user/temp/encoderpro-work:/work \
  -v /mnt/user/appdata/encoderpro/config.yaml:/config/config.yaml:ro \
  -v /mnt/user/appdata/encoderpro/db:/db \
  -v /mnt/user/appdata/encoderpro/logs:/logs \
  --restart unless-stopped \
  encoderpro-intel:latest dashboard

7. Access Dashboard:
   • Open: http://your-unraid-ip:5000

Unraid Template (Intel Arc)

Add Container Settings:

• Name: encoderpro-intel
• Repository: encoderpro-intel:latest
• Network Type: bridge

Environment Variables:

GPU_TYPE=intel
DASHBOARD_PORT=5000

Device Mapping (CRITICAL!):

/dev/dri → /dev/dri

Path Mappings:

/movies           → /mnt/user/movies (read-only)
/archive          → /mnt/user/archive/movies (read-write)
/work             → /mnt/user/temp/encoderpro-work (read-write)
/config/config.yaml → /mnt/user/appdata/encoderpro/config-intel.yaml (read-only)
/db               → /mnt/user/appdata/encoderpro/db (read-write)
/logs             → /mnt/user/appdata/encoderpro/logs (read-write)

Port:

5000:5000

See INTEL-ARC-GUIDE.md for detailed Intel Arc setup

---

CPU-Only Setup

Installation

1. Create app directory:

mkdir -p /mnt/user/appdata/encoderpro
cd /mnt/user/appdata/encoderpro

2. Build Docker image:

docker build -t encoderpro:latest .

3. Fix directory permissions:

# Create and set ownership (UID 1000)
mkdir -p /mnt/user/appdata/encoderpro/{db,logs}
mkdir -p /mnt/user/temp/encoderpro-work
mkdir -p /mnt/user/archive/movies
chown -R 1000:1000 /mnt/user/appdata/encoderpro/{db,logs}
chown -R 1000:1000 /mnt/user/temp/encoderpro-work
chown -R 1000:1000 /mnt/user/archive/movies

4. Copy config:

cp config-cpu.yaml config.yaml
# Edit to use CPU profiles (cpu_x265, cpu_x264)
nano config.yaml

5. Start Container:

docker run -d \
  --name encoderpro-dashboard-cpu \
  -e DASHBOARD_HOST=0.0.0.0 \
  -e DASHBOARD_PORT=5000 \
  -p 5000:5000 \
  -v /mnt/user/movies:/movies \
  -v /mnt/user/archive/movies:/archive \
  -v /mnt/user/temp/encoderpro-work:/work \
  -v /mnt/user/appdata/encoderpro/config.yaml:/config/config.yaml:ro \
  -v /mnt/user/appdata/encoderpro/db:/db \
  -v /mnt/user/appdata/encoderpro/logs:/logs \
  --restart unless-stopped \
  encoderpro:latest dashboard

Unraid Template (CPU)

Add Container Settings:

• Name: encoderpro-cpu
• Repository: encoderpro:latest
• Network Type: bridge

Environment Variables:

DASHBOARD_PORT=5000

Path Mappings:

/movies           → /mnt/user/movies (read-only)
/archive          → /mnt/user/archive/movies (read-write)
/work             → /mnt/user/temp/encoderpro-work (read-write)
/config/config.yaml → /mnt/user/appdata/encoderpro/config.yaml (read-only)
/db               → /mnt/user/appdata/encoderpro/db (read-write)
/logs             → /mnt/user/appdata/encoderpro/logs (read-write)

Port:

5000:5000

---

Configuration

Basic Config (All Platforms)

Edit /mnt/user/appdata/encoderpro/config.yaml:

# Paths are already set for Docker - no changes needed
movies_dir: /movies
archive_dir: /archive
work_dir: /work
state_db: /db/state.db
log_dir: /logs

# Adjust concurrent workers based on your hardware
parallel:
  max_workers: 2  # NVIDIA: 1-2, Intel Arc: 2-4, CPU: 2-4
  gpu_slots: 1    # NVIDIA: 1-2, Intel Arc: 2-3, CPU: N/A

# Choose your encoding profile
profiles:
  default: balanced_gpu  # NVIDIA: balanced_gpu
                         # Intel: balanced_qsv
                         # CPU: balanced_cpu

Recommended Profiles

NVIDIA:

profiles:
  default: balanced_gpu  # H.265, good quality/speed

Intel Arc:

profiles:
  default: balanced_qsv  # H.265, good quality/speed
  # Or for best compression:
  default: av1_qsv       # AV1, excellent compression (Arc A-Series only)

CPU:

profiles:
  default: balanced_cpu  # H.265, slow but good quality

---

Usage

Web Dashboard

1. Navigate to http://your-unraid-ip:5000
2. View statistics, browse files, manage jobs
3. Start/stop encoding from the UI

Command Line

Scan your library:

# NVIDIA
docker exec encoderpro-dashboard python3 /app/reencode.py -c /config/config.yaml --scan-only

# Intel
docker exec encoderpro-dashboard-intel python3 /app/reencode.py -c /config/config.yaml --scan-only

# CPU
docker exec encoderpro-dashboard-cpu python3 /app/reencode.py -c /config/config.yaml --scan-only

View statistics:

docker exec <container-name> python3 /app/reencode.py -c /config/config.yaml --stats

Start processing:

docker exec <container-name> python3 /app/reencode.py -c /config/config.yaml

Scheduled Processing (User Scripts)

Create a new User Script in Unraid:

#!/bin/bash
# Run encoderpro processing

cd /mnt/user/appdata/encoderpro

# For NVIDIA:
docker exec encoderpro-dashboard python3 /app/reencode.py -c /config/config.yaml

# For Intel:
# docker exec encoderpro-dashboard-intel python3 /app/reencode.py -c /config/config.yaml

# For CPU:
# docker exec encoderpro-dashboard-cpu python3 /app/reencode.py -c /config/config.yaml

Schedule: Daily or Weekly

---

Monitoring

View Logs

Real-time:

docker logs -f <container-name>

Log files:

tail -f /mnt/user/appdata/encoderpro/logs/encoderpro.log

Check Progress

Web Dashboard:

• Go to http://your-unraid-ip:5000
• View real-time statistics

Command line:

docker exec <container-name> python3 /app/reencode.py -c /config/config.yaml --stats

---

Troubleshooting

NVIDIA: GPU Not Found

# Check NVIDIA plugin installed
nvidia-smi

# Verify container can see GPU
docker exec encoderpro-dashboard nvidia-smi

Intel: GPU Not Found

# Check device exists
ls -la /dev/dri/

# Check container can see device
docker exec encoderpro-dashboard-intel ls -la /dev/dri/

# Verify QSV encoders
docker exec encoderpro-dashboard-intel ffmpeg -encoders | grep qsv

Read-Only File System Error

Symptom: [Errno 30] Read-only file system: '/movies/...'

Cause: The /movies directory was mounted as read-only (:ro), but encoderPro needs write access to replace files after encoding.

Why Write Access is Needed:

1. Original file is read from /movies/movie.mkv
2. Encoded to /work/movie.mkv.tmp
3. Original moved to /archive/movie.mkv (backup)
4. Encoded file moved to /movies/movie.mkv (replacement)

Fix: Remove the :ro flag from the movies mount:

# Stop and remove container
docker stop encoderpro-dashboard-intel && docker rm encoderpro-dashboard-intel

# Restart with read-write movies mount (no :ro)
docker run -d \
  --name encoderpro-dashboard-intel \
  --device=/dev/dri:/dev/dri \
  -e GPU_TYPE=intel \
  -p 5000:5000 \
  -v /mnt/user/movies:/movies \
  -v /mnt/user/archive/movies:/archive \
  -v /mnt/user/appdata/encoderpro/config.yaml:/config/config.yaml:ro \
  -v /mnt/user/appdata/encoderpro/db:/db \
  -v /mnt/user/appdata/encoderpro/logs:/logs \
  --restart unless-stopped \
  encoderpro-intel:latest dashboard

Note: The original files are safely backed up to /archive before replacement, so write access to /movies is safe.

Permission Denied Errors

Symptom: PermissionError: [Errno 13] Permission denied: '/logs/reencode.log'

Cause: Container runs as non-root user (UID 1000) for security, but directories aren't owned by that user.

Fix:

# Fix all directory permissions
chown -R 1000:1000 /mnt/user/appdata/encoderpro/db
chown -R 1000:1000 /mnt/user/appdata/encoderpro/logs
chown -R 1000:1000 /mnt/user/temp/encoderpro-work
chown -R 1000:1000 /mnt/user/archive/movies
chown -R 1000:1000 /mnt/user/movies

# Restart container
docker restart encoderpro-dashboard-intel
# OR (for NVIDIA)
docker restart encoderpro-dashboard
# OR (for CPU)
docker restart encoderpro-dashboard-cpu

Files Stuck in "Processing" State

Symptom: After stopping container, files show as "processing" and can't be selected

Fix:

1. Automatic: Just refresh the dashboard page - cleanup runs automatically
2. Manual: Click the "🔧 Reset Stuck" button in the dashboard
3. Result: Files are marked as "failed" and can be retried

Note: This happens when the container stops while encoding. The system automatically detects and fixes this on startup or dashboard refresh.

Container Won't Start

# Check logs
docker logs <container-name>

# Rebuild image
cd /mnt/user/appdata/encoderpro
docker build -t encoderpro:latest .  # or encoderpro-intel:latest

Encoding Too Slow

1. Reduce max_workers in config.yaml
2. Use faster profile (fast_gpu, fast_qsv, fast_cpu)
3. Ensure work_dir is on SSD

Out of Space

# Check work directory
du -sh /mnt/user/temp/encoderpro-work/

# Clean up stale files
rm -rf /mnt/user/temp/encoderpro-work/*

---

Performance Guide

NVIDIA Performance

GPU Series	Expected Speed (1080p)	Recommended Workers
RTX 4000	200-350 fps	2
RTX 3000	150-300 fps	1-2
RTX 2000	100-250 fps	1-2
GTX 1000	80-200 fps	1

Intel Arc Performance

GPU Model	Expected Speed (1080p)	Recommended Workers
A770	150-250 fps	3-4
A750	130-220 fps	3
A380	100-180 fps	2-3
A310	80-150 fps	2

CPU Performance

CPU Cores	Expected Speed (1080p)	Recommended Workers
16+ cores	5-10 fps	3-4
8-12 cores	3-7 fps	2-3
4-6 cores	2-5 fps	1-2

---

Updating

cd /mnt/user/appdata/encoderpro

# Pull latest code
git pull  # or re-download files

# Rebuild image
docker build -t encoderpro:latest .
# or for Intel:
docker build -f Dockerfile.intel -t encoderpro-intel:latest .

# Stop and remove old container
docker stop encoderpro-dashboard && docker rm encoderpro-dashboard
# OR for Intel:
docker stop encoderpro-dashboard-intel && docker rm encoderpro-dashboard-intel
# OR for CPU:
docker stop encoderpro-dashboard-cpu && docker rm encoderpro-dashboard-cpu

# Start new container (use the docker run command from the setup section above)

---

Support

• General Setup: See README-phase3.md
• Intel Arc: See INTEL-ARC-GUIDE.md
• Docker: See README-docker.md
• Dashboard: See DASHBOARD-GUIDE.md

---

Quick Reference

Build Images:

docker build -t encoderpro:latest .                              # NVIDIA/CPU
docker build -f Dockerfile.intel -t encoderpro-intel:latest .   # Intel

Start Services:

# NVIDIA
docker start encoderpro-dashboard

# Intel Arc
docker start encoderpro-dashboard-intel

# CPU only
docker start encoderpro-dashboard-cpu

Process Files:

docker exec <container> python3 /app/reencode.py -c /config/config.yaml

View Stats:

docker exec <container> python3 /app/reencode.py -c /config/config.yaml --stats

Web Dashboard:

http://your-unraid-ip:5000