encoderPro/QUALITY-CHECKING.md

# Quality Checking Feature

## Overview

encoderPro now includes intelligent quality analysis that detects source video quality and warns you if encoding will degrade quality to a noticeable degree. This prevents accidental quality loss when re-encoding high-quality source material.

---

## Key Features

### 1. **Automatic Quality Detection**
- Analyzes source video bitrate, resolution, codec, and FPS
- Calculates quality score (0-100) based on bitrate per pixel
- Detects HDR content
- Accounts for codec efficiency (H.265 vs H.264 vs AV1)

### 2. **Pre-Encoding Quality Comparison**
- Estimates target encoding bitrate based on your CRF settings
- Compares source quality to estimated target quality
- Warns if quality drop exceeds configurable thresholds
- Provides detailed quality metrics for decision making

### 3. **Processed File Tracking**
- Database tracks all processed files
- Automatically skips already-encoded files on re-runs
- Maintains processing history with quality metrics
- Supports manual reset for re-processing

### 4. **Configurable Behavior**
- Enable/disable quality checking
- Adjustable warning and error thresholds
- Option to skip degraded files automatically
- User prompts for manual confirmation

---

## Configuration

Add this section to your `config.yaml`:

```yaml
quality_check:
  # Enable pre-encoding quality analysis
  enabled: true

  # Warning threshold - warn if quality will drop by this many points (0-100 scale)
  warn_threshold: 10.0

  # Error threshold - fail/skip if quality will drop by this many points
  error_threshold: 20.0

  # Automatically skip files where encoding would degrade quality
  skip_on_degradation: false

  # Prompt user for confirmation when warnings detected (CLI only)
  prompt_on_warning: true
```

### Configuration Options Explained

| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `enabled` | boolean | `true` | Enable/disable quality checking entirely |
| `warn_threshold` | float | `10.0` | Quality score drop that triggers a warning |
| `error_threshold` | float | `20.0` | Quality score drop that triggers an error |
| `skip_on_degradation` | boolean | `false` | Auto-skip files with quality degradation |
| `prompt_on_warning` | boolean | `true` | Ask user for confirmation on warnings |

---

## How Quality Scores Work

### Quality Score (0-100)

The quality checker calculates a score based on **bits per pixel per frame**:

```
Quality Score = f(bitrate, resolution, fps, codec_efficiency)
```

**Score Ranges:**
- **95-100:** Near-lossless / Exceptional quality
- **85-95:** Excellent / Archival quality
- **70-85:** Good / Visually transparent
- **50-70:** Acceptable / Minor compression visible
- **0-50:** Poor / Heavy compression artifacts

### Codec Efficiency Multipliers

The quality checker accounts for codec efficiency:

| Codec | Multiplier | Notes |
|-------|------------|-------|
| AV1 | 1.8x | Most efficient codec |
| H.265/HEVC | 1.5x | ~50% better than H.264 |
| H.264/AVC | 1.0x | Baseline reference |
| MPEG-2 | 0.5x | Older, less efficient |
| MPEG-4 | 0.7x | Older codec |

**Example:** A video at 5 Mbps H.265 has similar quality to 7.5 Mbps H.264.

### Bits Per Pixel Ranges

| BPP Range | Quality | Typical Use Case |
|-----------|---------|------------------|
| > 0.5 | Near-lossless (95-100) | Professional archival |
| 0.3 - 0.5 | Excellent (85-95) | High-quality archival |
| 0.2 - 0.3 | Good (70-85) | **Sweet spot for home media** |
| 0.1 - 0.2 | Acceptable (50-70) | Streaming services |
| < 0.1 | Poor (0-50) | Heavy compression |

---

## Usage Examples

### Example 1: Analyze Video Quality

```bash
python3 quality_checker.py /movies/example.mkv
```

**Output:**
```
Video Quality Analysis:
  Resolution: 1920x1080
  Bitrate: 8.5 Mbps
  Codec: h264
  FPS: 23.98
  HDR: No
  Quality Score: 72.3/100
```

### Example 2: Check Before Encoding

```python
from quality_checker import QualityChecker
from pathlib import Path

checker = QualityChecker()

# Define your encoding profile
profile = {
    'encoder': 'nvidia_nvenc_h265',
    'quality': 21  # CRF value
}

# Check quality before encoding
result = checker.check_before_encode(
    Path('/movies/example.mkv'),
    profile,
    warn_threshold=10.0,
    error_threshold=20.0
)

if result['ok']:
    print("✅ Safe to encode")
elif result['warning']:
    print(f"⚠️ Warning: {result['message']}")
elif result['error']:
    print(f"❌ Error: {result['message']}")
```

### Example 3: Batch Analysis

See `example_quality_check.py` for comprehensive examples including:
- Single file analysis
- Quality degradation checks
- Comprehensive pre-encode checks
- Batch directory analysis

---

## Understanding Quality Degradation

### What Causes Quality Degradation?

Encoding can degrade quality when:

1. **Source is already high quality**
   - Source: 95/100 (near-lossless BluRay rip at 25 Mbps)
   - Target: 75/100 (H.265 CRF 21 at 8 Mbps)
   - **Drop:** 20 points ⚠️

2. **Target bitrate is too low**
   - Source: 4K movie at 40 Mbps
   - Target: 4K at 10 Mbps (too aggressive)
   - **Result:** Visible compression artifacts

3. **Re-encoding already compressed content**
   - Source: Web-DL already at CRF 23
   - Target: Re-encode at CRF 21
   - **Result:** Quality loss from generation loss

### When to Proceed Despite Warnings

**Safe to proceed:**
- Source quality 50-70 (already compressed)
- Quality drop < 10 points
- Testing settings on sample files
- Source is a screen recording or low-quality capture

**Consider skipping:**
- Source quality > 90 (near-lossless)
- Quality drop > 20 points
- Archival content you want to preserve
- BluRay remuxes or untouched sources

---

## Real-World Examples

### Example 1: BluRay Remux (Skip Recommended)

```
Source Quality Analysis:
  Resolution: 1920x1080
  Bitrate: 28.5 Mbps (H.264)
  Quality Score: 96/100 (Near-lossless)

Target Settings (CRF 21 H.265):
  Estimated Bitrate: 7.2 Mbps
  Target Quality: 73/100

⚠️ WARNING: Quality will drop by 23 points
❌ Recommendation: SKIP - Source is too high quality
```

**Action:** Skip encoding or use CRF 18-19 for archival quality.

---

### Example 2: Web-DL (Safe to Encode)

```
Source Quality Analysis:
  Resolution: 1920x1080
  Bitrate: 6.2 Mbps (H.264)
  Quality Score: 68/100 (Good)

Target Settings (CRF 21 H.265):
  Estimated Bitrate: 5.8 Mbps
  Target Quality: 71/100

✅ OK: Quality will improve by 3 points
✅ Recommendation: PROCEED - Safe to encode
```

**Action:** Proceed with encoding. Will remove subtitles without quality loss.

---

### Example 3: Already Encoded (Warning)

```
Source Quality Analysis:
  Resolution: 1920x1080
  Bitrate: 4.8 Mbps (H.265)
  Quality Score: 65/100 (Acceptable)

Target Settings (CRF 21 H.265):
  Estimated Bitrate: 5.2 Mbps
  Target Quality: 68/100

⚠️ WARNING: Re-encoding already compressed H.265
⚠️ Recommendation: Consider CRF 19-20 to avoid generation loss
```

**Action:** Consider lower CRF or skip. Source is already H.265.

---

### Example 4: Low Quality Source (Safe)

```
Source Quality Analysis:
  Resolution: 1920x1080
  Bitrate: 2.8 Mbps (H.264)
  Quality Score: 52/100 (Acceptable)

Target Settings (CRF 21 H.265):
  Estimated Bitrate: 5.8 Mbps
  Target Quality: 71/100

✅ OK: Quality will improve by 19 points
✅ Recommendation: PROCEED - Encoding will improve quality
```

**Action:** Proceed. H.265 encoding will improve perceived quality.

---

## HDR Content Detection

The quality checker automatically detects HDR content:

### HDR Detection Criteria

1. **Transfer characteristics:**
   - SMPTE 2084 (HDR10)
   - ARIB STD-B67 (HLG)

2. **Color primaries:**
   - BT.2020 color space

3. **Metadata tags:**
   - Any tag containing "HDR"

### HDR Warning

If HDR content is detected and `hdr_handling` is not set to `preserve`:

```
⚠️ HDR content detected but HDR handling is not set to 'preserve'
```

**Action:** Update your profile:
```yaml
profiles:
  your_profile:
    hdr_handling: preserve
```

---

## Processed File Tracking

### How It Works

1. **First run:** All files scanned and added to database as `pending`
2. **Encoding:** Files marked as `processing` → `completed` or `failed`
3. **Second run:** Completed files automatically skipped
4. **Re-processing:** Reset state to `pending` if needed

### Database Schema

```sql
CREATE TABLE files (
    id INTEGER PRIMARY KEY,
    filepath TEXT UNIQUE NOT NULL,
    state TEXT NOT NULL,  -- pending/processing/completed/failed/skipped
    profile_name TEXT,
    encoder_used TEXT,
    encode_time_seconds REAL,
    fps REAL,
    original_size INTEGER,
    encoded_size INTEGER,
    source_quality_score REAL,  -- NEW
    target_quality_score REAL,  -- NEW
    created_at TIMESTAMP,
    updated_at TIMESTAMP
);
```

### Manually Reset Files

```bash
# Reset specific file
python3 dbmanage.py --reset-file /movies/example.mkv

# Reset all files
python3 dbmanage.py --reset-all

# Reset only failed files
python3 dbmanage.py --reset-failed
```

---

## Command-Line Options

### Check Quality Only (No Encoding)

```bash
# Analyze single file
python3 quality_checker.py /movies/example.mkv

# Batch analyze directory
python3 example_quality_check.py /movies/
```

### Override Quality Check

```bash
# Disable quality check for this run
python3 reencode.py -c config.yaml --no-quality-check

# Force encode despite warnings
python3 reencode.py -c config.yaml --force
```

### Skip Processed Files

```bash
# Skip already completed files (default behavior)
python3 reencode.py -c config.yaml

# Re-process all files (ignore database)
python3 reencode.py -c config.yaml --reprocess-all
```

---

## Best Practices

### 1. **Start with Quality Analysis**

Before encoding your entire library:

```bash
# Analyze quality distribution
python3 example_quality_check.py /movies/ > quality_report.txt
```

Review the report to understand your source quality distribution.

---

### 2. **Use Appropriate Thresholds**

**Conservative (Preserve Quality):**
```yaml
quality_check:
  warn_threshold: 5.0    # Warn on small drops
  error_threshold: 10.0  # Error on moderate drops
  skip_on_degradation: true
```

**Balanced (Recommended):**
```yaml
quality_check:
  warn_threshold: 10.0
  error_threshold: 20.0
  skip_on_degradation: false
```

**Aggressive (Maximum Compression):**
```yaml
quality_check:
  warn_threshold: 20.0
  error_threshold: 30.0
  skip_on_degradation: false
```

---

### 3. **Quality-Based Profiles**

Use resolution rules to apply different profiles based on source quality:

```yaml
advanced:
  resolution_rules:
    enabled: true
    quality_rules:
      - min_quality: 90    # Near-lossless sources
        profile: quality_gpu  # Use high-quality preset
      - min_quality: 70
        profile: sweetspot_gpu  # Balanced
      - min_quality: 50
        profile: balanced_gpu   # Already compressed
      - min_quality: 0
        profile: fast_gpu       # Low quality sources
```

---

### 4. **Monitor Processing Logs**

Quality checks are logged:

```
2025-01-15 10:30:22 - INFO - Analyzing quality: example.mkv
2025-01-15 10:30:23 - INFO - Source quality: 72.3/100
2025-01-15 10:30:23 - INFO - Target quality: 75.1/100
2025-01-15 10:30:23 - INFO - ✅ Quality check passed
```

Review logs to identify patterns and adjust settings.

---

## Troubleshooting

### Quality Check Fails

**Problem:** `Failed to analyze source video quality`

**Solutions:**
1. Check ffprobe is installed: `ffprobe -version`
2. Verify file is readable: `ls -la /path/to/file.mkv`
3. Check file is valid video: `ffprobe /path/to/file.mkv`
4. Disable quality check temporarily:
   ```yaml
   quality_check:
     enabled: false
   ```

---

### Inaccurate Quality Scores

**Problem:** Quality scores don't match expectations

**Explanation:**
- Quality scores are **estimates** based on bitrate per pixel
- Different content has different complexity
  - Simple animation: Lower bitrate, high quality
  - Grain-heavy film: Higher bitrate needed
- Scores are relative, not absolute

**Solution:** Use thresholds as guidelines, not absolute rules.

---

### Too Many Warnings

**Problem:** Getting warnings on files you want to encode

**Solutions:**

1. **Increase thresholds:**
   ```yaml
   quality_check:
     warn_threshold: 15.0  # From 10.0
   ```

2. **Disable warnings:**
   ```yaml
   quality_check:
     prompt_on_warning: false
   ```

3. **Disable quality check:**
   ```yaml
   quality_check:
     enabled: false
   ```

---

## API Reference

See `quality_checker.py` for full API documentation.

### QualityChecker Class

```python
class QualityChecker:
    def analyze_quality(self, filepath: Path) -> Optional[VideoQuality]:
        """Analyze video quality metrics"""

    def will_degrade_quality(
        self,
        source_quality: VideoQuality,
        target_bitrate: int,
        target_codec: str,
        threshold: float = 10.0
    ) -> Tuple[bool, str]:
        """Check if encoding will significantly degrade quality"""

    def check_before_encode(
        self,
        filepath: Path,
        profile: Dict,
        warn_threshold: float = 10.0,
        error_threshold: float = 20.0
    ) -> Dict:
        """Comprehensive quality check before encoding"""

    def estimate_target_bitrate(
        self,
        profile: Dict,
        resolution: Tuple[int, int],
        fps: float
    ) -> int:
        """Estimate target bitrate based on profile settings"""
```

---

## Summary

The quality checking feature provides:

✅ **Intelligent quality analysis** - Understand your source material
✅ **Degradation detection** - Avoid accidental quality loss
✅ **Processed file tracking** - Skip already-encoded files
✅ **Configurable behavior** - Customize to your needs
✅ **HDR detection** - Preserve HDR metadata
✅ **Detailed logging** - Monitor quality decisions

**Best Use Cases:**
- Large mixed-quality libraries
- Preserving high-quality sources
- Avoiding re-encoding already compressed files
- Understanding quality impact before encoding

**When to Disable:**
- All sources are known low quality
- Testing encoding settings
- Processing screen recordings
- Time-sensitive batch jobs