encoderPro/UNRAID-DEPLOYMENT.md

# encoderPro - Complete Deployment Guide

**THE OFFICIAL SETUP GUIDE - START HERE**

This is the complete deployment guide for encoderPro. It covers installation on Unraid with support for NVIDIA GPUs, Intel Arc GPUs, or CPU-only encoding.

> **Note:** This guide uses direct `docker run` commands for Unraid deployment. The `docker-compose.yml` file is optional and not required for Unraid.

---

## Quick Start for Unraid

This guide shows you how to deploy encoderPro on Unraid with support for NVIDIA GPUs, Intel Arc GPUs, or CPU-only encoding.

## Choose Your Hardware

### Option 1: NVIDIA GPU (NVENC)
- Best for: RTX 2000-4000 series owners
- Speed: Very fast (100-300 fps)
- Codecs: H.264, H.265
- Follow: [NVIDIA Setup](#nvidia-gpu-setup)

### Option 2: Intel Arc GPU (QSV)
- Best for: Arc A-Series or Intel integrated graphics users
- Speed: Very fast (80-250 fps)
- Codecs: H.264, H.265, **AV1** (Arc only!)
- Follow: [Intel Arc Setup](#intel-arc-setup)

### Option 3: CPU Only
- Best for: No GPU or testing
- Speed: Slow (2-10 fps)
- Codecs: All software codecs
- Follow: [CPU Setup](#cpu-only-setup)

---

## NVIDIA GPU Setup

### Prerequisites

1. **Install NVIDIA Plugin:**
   - Go to Unraid Plugins
   - Install "Nvidia-Driver" plugin from Community Applications
   - Reboot

2. **Verify GPU:**
```bash
nvidia-smi
```

### Installation

1. **Create app directory:**
```bash
mkdir -p /mnt/user/appdata/encoderpro
cd /mnt/user/appdata/encoderpro
```

2. **Copy project files:**
   - Place all encoderPro files in `/mnt/user/appdata/encoderpro`

3. **Build Docker image:**
```bash
docker build -t encoderpro:latest .
```

4. **Fix directory permissions:**
```bash
# Create and set ownership (UID 1000)
mkdir -p /mnt/user/appdata/encoderpro/{db,logs}
mkdir -p /mnt/user/temp/encoderpro-work
mkdir -p /mnt/user/archive/movies
chown -R 1000:1000 /mnt/user/appdata/encoderpro/{db,logs}
chown -R 1000:1000 /mnt/user/temp/encoderpro-work
chown -R 1000:1000 /mnt/user/archive/movies
```

5. **Copy config:**
```bash
cp config-nvidia.yaml config.yaml
nano config.yaml  # Adjust paths if needed
```

6. **Start Container:**
```bash
docker run -d \
  --name encoderpro-dashboard \
  --runtime=nvidia \
  -e NVIDIA_VISIBLE_DEVICES=all \
  -e NVIDIA_DRIVER_CAPABILITIES=compute,video,utility \
  -e DASHBOARD_HOST=0.0.0.0 \
  -e DASHBOARD_PORT=5000 \
  -p 5000:5000 \
  -v /mnt/user/movies:/movies \
  -v /mnt/user/archive/movies:/archive \
  -v /mnt/user/temp/encoderpro-work:/work \
  -v /mnt/user/appdata/encoderpro/config.yaml:/config/config.yaml:ro \
  -v /mnt/user/appdata/encoderpro/db:/db \
  -v /mnt/user/appdata/encoderpro/logs:/logs \
  --restart unless-stopped \
  encoderpro:latest dashboard
```

7. **Access Dashboard:**
   - Open: `http://your-unraid-ip:5000`

### Unraid Template (NVIDIA)

**Add Container Settings:**
- Name: `encoderpro-dashboard`
- Repository: `encoderpro:latest`
- Network Type: `bridge`
- Extra Parameters: `--runtime=nvidia`

**Environment Variables:**
```
NVIDIA_VISIBLE_DEVICES=all
NVIDIA_DRIVER_CAPABILITIES=compute,video,utility
DASHBOARD_PORT=5000
```

**Path Mappings:**
```
/movies           → /mnt/user/movies (read-only)
/archive          → /mnt/user/archive/movies (read-write)
/work             → /mnt/user/temp/encoderpro-work (read-write)
/config/config.yaml → /mnt/user/appdata/encoderpro/config.yaml (read-only)
/db               → /mnt/user/appdata/encoderpro/db (read-write)
/logs             → /mnt/user/appdata/encoderpro/logs (read-write)
```

**Port:**
```
5000:5000
```

---

## Intel Arc Setup

### Prerequisites

1. **Verify GPU detected:**
```bash
lspci | grep -i vga
ls -la /dev/dri/  # Should show renderD128 and card0
```

2. **No special Unraid plugins needed** - drivers install in container

### Installation

1. **Create app directory:**
```bash
mkdir -p /mnt/user/appdata/encoderpro
cd /mnt/user/appdata/encoderpro
```

2. **Copy project files:**
   - Place all encoderPro files in `/mnt/user/appdata/encoderpro`

3. **Build Intel Docker image:**
```bash
docker build -f Dockerfile.intel -t encoderpro-intel:latest .
```
   *(This takes 5-10 minutes - installs Intel drivers)*

4. **Fix directory permissions (CRITICAL!):**
```bash
# Create directories
mkdir -p /mnt/user/appdata/encoderpro/db
mkdir -p /mnt/user/appdata/encoderpro/logs
mkdir -p /mnt/user/temp/encoderpro-work
mkdir -p /mnt/user/archive/movies

# Set ownership to UID 1000 (container's encoder user)
chown -R 1000:1000 /mnt/user/appdata/encoderpro/db
chown -R 1000:1000 /mnt/user/appdata/encoderpro/logs
chown -R 1000:1000 /mnt/user/temp/encoderpro-work
chown -R 1000:1000 /mnt/user/archive/movies
```

5. **Copy Intel config:**
```bash
cp config-intel.yaml config.yaml
nano config.yaml  # Adjust settings for your Arc model
```

6. **Start Container:**
```bash
docker run -d \
  --name encoderpro-dashboard-intel \
  --device=/dev/dri:/dev/dri \
  -e GPU_TYPE=intel \
  -e DASHBOARD_HOST=0.0.0.0 \
  -e DASHBOARD_PORT=5000 \
  -p 5000:5000 \
  -v /mnt/user/movies:/movies \
  -v /mnt/user/archive/movies:/archive \
  -v /mnt/user/temp/encoderpro-work:/work \
  -v /mnt/user/appdata/encoderpro/config.yaml:/config/config.yaml:ro \
  -v /mnt/user/appdata/encoderpro/db:/db \
  -v /mnt/user/appdata/encoderpro/logs:/logs \
  --restart unless-stopped \
  encoderpro-intel:latest dashboard
```

7. **Access Dashboard:**
   - Open: `http://your-unraid-ip:5000`

### Unraid Template (Intel Arc)

**Add Container Settings:**
- Name: `encoderpro-intel`
- Repository: `encoderpro-intel:latest`
- Network Type: `bridge`

**Environment Variables:**
```
GPU_TYPE=intel
DASHBOARD_PORT=5000
```

**Device Mapping (CRITICAL!):**
```
/dev/dri → /dev/dri
```

**Path Mappings:**
```
/movies           → /mnt/user/movies (read-only)
/archive          → /mnt/user/archive/movies (read-write)
/work             → /mnt/user/temp/encoderpro-work (read-write)
/config/config.yaml → /mnt/user/appdata/encoderpro/config-intel.yaml (read-only)
/db               → /mnt/user/appdata/encoderpro/db (read-write)
/logs             → /mnt/user/appdata/encoderpro/logs (read-write)
```

**Port:**
```
5000:5000
```

**See [INTEL-ARC-GUIDE.md](INTEL-ARC-GUIDE.md) for detailed Intel Arc setup**

---

## CPU-Only Setup

### Installation

1. **Create app directory:**
```bash
mkdir -p /mnt/user/appdata/encoderpro
cd /mnt/user/appdata/encoderpro
```

2. **Build Docker image:**
```bash
docker build -t encoderpro:latest .
```

3. **Fix directory permissions:**
```bash
# Create and set ownership (UID 1000)
mkdir -p /mnt/user/appdata/encoderpro/{db,logs}
mkdir -p /mnt/user/temp/encoderpro-work
mkdir -p /mnt/user/archive/movies
chown -R 1000:1000 /mnt/user/appdata/encoderpro/{db,logs}
chown -R 1000:1000 /mnt/user/temp/encoderpro-work
chown -R 1000:1000 /mnt/user/archive/movies
```

4. **Copy config:**
```bash
cp config-cpu.yaml config.yaml
# Edit to use CPU profiles (cpu_x265, cpu_x264)
nano config.yaml
```

5. **Start Container:**
```bash
docker run -d \
  --name encoderpro-dashboard-cpu \
  -e DASHBOARD_HOST=0.0.0.0 \
  -e DASHBOARD_PORT=5000 \
  -p 5000:5000 \
  -v /mnt/user/movies:/movies \
  -v /mnt/user/archive/movies:/archive \
  -v /mnt/user/temp/encoderpro-work:/work \
  -v /mnt/user/appdata/encoderpro/config.yaml:/config/config.yaml:ro \
  -v /mnt/user/appdata/encoderpro/db:/db \
  -v /mnt/user/appdata/encoderpro/logs:/logs \
  --restart unless-stopped \
  encoderpro:latest dashboard
```

### Unraid Template (CPU)

**Add Container Settings:**
- Name: `encoderpro-cpu`
- Repository: `encoderpro:latest`
- Network Type: `bridge`

**Environment Variables:**
```
DASHBOARD_PORT=5000
```

**Path Mappings:**
```
/movies           → /mnt/user/movies (read-only)
/archive          → /mnt/user/archive/movies (read-write)
/work             → /mnt/user/temp/encoderpro-work (read-write)
/config/config.yaml → /mnt/user/appdata/encoderpro/config.yaml (read-only)
/db               → /mnt/user/appdata/encoderpro/db (read-write)
/logs             → /mnt/user/appdata/encoderpro/logs (read-write)
```

**Port:**
```
5000:5000
```

---

## Configuration

### Basic Config (All Platforms)

Edit `/mnt/user/appdata/encoderpro/config.yaml`:

```yaml
# Paths are already set for Docker - no changes needed
movies_dir: /movies
archive_dir: /archive
work_dir: /work
state_db: /db/state.db
log_dir: /logs

# Adjust concurrent workers based on your hardware
parallel:
  max_workers: 2  # NVIDIA: 1-2, Intel Arc: 2-4, CPU: 2-4
  gpu_slots: 1    # NVIDIA: 1-2, Intel Arc: 2-3, CPU: N/A

# Choose your encoding profile
profiles:
  default: balanced_gpu  # NVIDIA: balanced_gpu
                         # Intel: balanced_qsv
                         # CPU: balanced_cpu
```

### Recommended Profiles

**NVIDIA:**
```yaml
profiles:
  default: balanced_gpu  # H.265, good quality/speed
```

**Intel Arc:**
```yaml
profiles:
  default: balanced_qsv  # H.265, good quality/speed
  # Or for best compression:
  default: av1_qsv       # AV1, excellent compression (Arc A-Series only)
```

**CPU:**
```yaml
profiles:
  default: balanced_cpu  # H.265, slow but good quality
```

---

## Usage

### Web Dashboard

1. Navigate to `http://your-unraid-ip:5000`
2. View statistics, browse files, manage jobs
3. Start/stop encoding from the UI

### Command Line

**Scan your library:**
```bash
# NVIDIA
docker exec encoderpro-dashboard python3 /app/reencode.py -c /config/config.yaml --scan-only

# Intel
docker exec encoderpro-dashboard-intel python3 /app/reencode.py -c /config/config.yaml --scan-only

# CPU
docker exec encoderpro-dashboard-cpu python3 /app/reencode.py -c /config/config.yaml --scan-only
```

**View statistics:**
```bash
docker exec <container-name> python3 /app/reencode.py -c /config/config.yaml --stats
```

**Start processing:**
```bash
docker exec <container-name> python3 /app/reencode.py -c /config/config.yaml
```

### Scheduled Processing (User Scripts)

Create a new User Script in Unraid:

```bash
#!/bin/bash
# Run encoderpro processing

cd /mnt/user/appdata/encoderpro

# For NVIDIA:
docker exec encoderpro-dashboard python3 /app/reencode.py -c /config/config.yaml

# For Intel:
# docker exec encoderpro-dashboard-intel python3 /app/reencode.py -c /config/config.yaml

# For CPU:
# docker exec encoderpro-dashboard-cpu python3 /app/reencode.py -c /config/config.yaml
```

Schedule: `Daily` or `Weekly`

---

## Monitoring

### View Logs

**Real-time:**
```bash
docker logs -f <container-name>
```

**Log files:**
```bash
tail -f /mnt/user/appdata/encoderpro/logs/encoderpro.log
```

### Check Progress

**Web Dashboard:**
- Go to `http://your-unraid-ip:5000`
- View real-time statistics

**Command line:**
```bash
docker exec <container-name> python3 /app/reencode.py -c /config/config.yaml --stats
```

---

## Troubleshooting

### NVIDIA: GPU Not Found

```bash
# Check NVIDIA plugin installed
nvidia-smi

# Verify container can see GPU
docker exec encoderpro-dashboard nvidia-smi
```

### Intel: GPU Not Found

```bash
# Check device exists
ls -la /dev/dri/

# Check container can see device
docker exec encoderpro-dashboard-intel ls -la /dev/dri/

# Verify QSV encoders
docker exec encoderpro-dashboard-intel ffmpeg -encoders | grep qsv
```

### Read-Only File System Error

**Symptom:** `[Errno 30] Read-only file system: '/movies/...'`

**Cause:** The `/movies` directory was mounted as read-only (`:ro`), but encoderPro needs write access to replace files after encoding.

**Why Write Access is Needed:**
1. Original file is read from `/movies/movie.mkv`
2. Encoded to `/work/movie.mkv.tmp`
3. Original moved to `/archive/movie.mkv` (backup)
4. Encoded file moved to `/movies/movie.mkv` (replacement)

**Fix:**
Remove the `:ro` flag from the movies mount:
```bash
# Stop and remove container
docker stop encoderpro-dashboard-intel && docker rm encoderpro-dashboard-intel

# Restart with read-write movies mount (no :ro)
docker run -d \
  --name encoderpro-dashboard-intel \
  --device=/dev/dri:/dev/dri \
  -e GPU_TYPE=intel \
  -p 5000:5000 \
  -v /mnt/user/movies:/movies \
  -v /mnt/user/archive/movies:/archive \
  -v /mnt/user/appdata/encoderpro/config.yaml:/config/config.yaml:ro \
  -v /mnt/user/appdata/encoderpro/db:/db \
  -v /mnt/user/appdata/encoderpro/logs:/logs \
  --restart unless-stopped \
  encoderpro-intel:latest dashboard
```

**Note:** The original files are safely backed up to `/archive` before replacement, so write access to `/movies` is safe.

### Permission Denied Errors

**Symptom:** `PermissionError: [Errno 13] Permission denied: '/logs/reencode.log'`

**Cause:** Container runs as non-root user (UID 1000) for security, but directories aren't owned by that user.

**Fix:**
```bash
# Fix all directory permissions
chown -R 1000:1000 /mnt/user/appdata/encoderpro/db
chown -R 1000:1000 /mnt/user/appdata/encoderpro/logs
chown -R 1000:1000 /mnt/user/temp/encoderpro-work
chown -R 1000:1000 /mnt/user/archive/movies
chown -R 1000:1000 /mnt/user/movies

# Restart container
docker restart encoderpro-dashboard-intel
# OR (for NVIDIA)
docker restart encoderpro-dashboard
# OR (for CPU)
docker restart encoderpro-dashboard-cpu
```

### Files Stuck in "Processing" State

**Symptom:** After stopping container, files show as "processing" and can't be selected

**Fix:**
1. **Automatic:** Just refresh the dashboard page - cleanup runs automatically
2. **Manual:** Click the "🔧 Reset Stuck" button in the dashboard
3. **Result:** Files are marked as "failed" and can be retried

**Note:** This happens when the container stops while encoding. The system automatically detects and fixes this on startup or dashboard refresh.

### Container Won't Start

```bash
# Check logs
docker logs <container-name>

# Rebuild image
cd /mnt/user/appdata/encoderpro
docker build -t encoderpro:latest .  # or encoderpro-intel:latest
```

### Encoding Too Slow

1. Reduce `max_workers` in config.yaml
2. Use faster profile (fast_gpu, fast_qsv, fast_cpu)
3. Ensure work_dir is on SSD

### Out of Space

```bash
# Check work directory
du -sh /mnt/user/temp/encoderpro-work/

# Clean up stale files
rm -rf /mnt/user/temp/encoderpro-work/*
```

---

## Performance Guide

### NVIDIA Performance

| GPU Series | Expected Speed (1080p) | Recommended Workers |
|------------|------------------------|---------------------|
| RTX 4000   | 200-350 fps            | 2                   |
| RTX 3000   | 150-300 fps            | 1-2                 |
| RTX 2000   | 100-250 fps            | 1-2                 |
| GTX 1000   | 80-200 fps             | 1                   |

### Intel Arc Performance

| GPU Model | Expected Speed (1080p) | Recommended Workers |
|-----------|------------------------|---------------------|
| A770      | 150-250 fps            | 3-4                 |
| A750      | 130-220 fps            | 3                   |
| A380      | 100-180 fps            | 2-3                 |
| A310      | 80-150 fps             | 2                   |

### CPU Performance

| CPU Cores | Expected Speed (1080p) | Recommended Workers |
|-----------|------------------------|---------------------|
| 16+ cores | 5-10 fps               | 3-4                 |
| 8-12 cores| 3-7 fps                | 2-3                 |
| 4-6 cores | 2-5 fps                | 1-2                 |

---

## Updating

```bash
cd /mnt/user/appdata/encoderpro

# Pull latest code
git pull  # or re-download files

# Rebuild image
docker build -t encoderpro:latest .
# or for Intel:
docker build -f Dockerfile.intel -t encoderpro-intel:latest .

# Stop and remove old container
docker stop encoderpro-dashboard && docker rm encoderpro-dashboard
# OR for Intel:
docker stop encoderpro-dashboard-intel && docker rm encoderpro-dashboard-intel
# OR for CPU:
docker stop encoderpro-dashboard-cpu && docker rm encoderpro-dashboard-cpu

# Start new container (use the docker run command from the setup section above)
```

---

## Support

- **General Setup**: See [README-phase3.md](README-phase3.md)
- **Intel Arc**: See [INTEL-ARC-GUIDE.md](INTEL-ARC-GUIDE.md)
- **Docker**: See [README-docker.md](README-docker.md)
- **Dashboard**: See [DASHBOARD-GUIDE.md](DASHBOARD-GUIDE.md)

---

## Quick Reference

**Build Images:**
```bash
docker build -t encoderpro:latest .                              # NVIDIA/CPU
docker build -f Dockerfile.intel -t encoderpro-intel:latest .   # Intel
```

**Start Services:**
```bash
# NVIDIA
docker start encoderpro-dashboard

# Intel Arc
docker start encoderpro-dashboard-intel

# CPU only
docker start encoderpro-dashboard-cpu
```

**Process Files:**
```bash
docker exec <container> python3 /app/reencode.py -c /config/config.yaml
```

**View Stats:**
```bash
docker exec <container> python3 /app/reencode.py -c /config/config.yaml --stats
```

**Web Dashboard:**
```
http://your-unraid-ip:5000
```