encoderPro/README.md

# encoderPro

**GPU-Accelerated Media Re-Encoding with Web Dashboard**

Modern, intelligent video encoding system with quality checking, state tracking, and a beautiful web interface for monitoring and control.

---

## Features

### Core Encoding
- **Smart Quality Detection** - Analyzes source quality and warns before degradation
- **GPU Acceleration** - NVIDIA NVENC, Intel QSV, or CPU encoding
- **State Tracking** - SQLite database tracks all processed files
- **Subtitle Detection** - Only processes files with subtitle streams
- **Resume Capability** - Interrupt and resume without losing progress
- **Profile-Based Encoding** - Pre-configured quality profiles for different use cases

### Web Dashboard
- **Real-Time Monitoring** - Live stats, progress, and activity feed
- **Job Control** - Start, stop, and manage encoding jobs
- **File Management** - Browse files, select specific movies to re-encode
- **Profile Selection** - Choose encoding profiles from the UI
- **Quality Analysis** - View file details, quality scores, and savings
- **System Health** - Monitor GPU/CPU usage, disk space, encoder status

### Quality Features
- **Pre-Encode Analysis** - Calculate quality scores before encoding
- **Degradation Detection** - Warn if encoding will lower quality
- **HDR Detection** - Identify HDR content for special handling
- **Bitrate Analysis** - Calculate bits-per-pixel for quality assessment
- **Skip Protection** - Optionally skip files that would degrade

---

## Quick Start

**For complete setup instructions, see [UNRAID-DEPLOYMENT.md](UNRAID-DEPLOYMENT.md)**

The deployment guide includes:
- NVIDIA GPU setup (RTX series)
- Intel Arc GPU setup (A-Series, integrated graphics)
- CPU-only setup
- Directory permissions (critical!)
- Troubleshooting common issues
- Performance tuning

### Quick Example (Intel Arc)

```bash
# Build image
docker build -f Dockerfile.intel -t encoderpro-intel:latest .

# Fix permissions (REQUIRED!)
chown -R 1000:1000 /mnt/user/appdata/encoderpro/{db,logs}
chown -R 1000:1000 /mnt/user/temp/encoderpro-work
chown -R 1000:1000 /mnt/user/archive/movies

# Run dashboard
docker run -d \
  --name encoderpro-dashboard-intel \
  --device=/dev/dri:/dev/dri \
  -e GPU_TYPE=intel \
  -p 5000:5000 \
  -v /mnt/user/movies:/movies \
  -v /mnt/user/archive/movies:/archive \
  -v /mnt/user/appdata/encoderpro/config.yaml:/config/config.yaml:ro \
  -v /mnt/user/appdata/encoderpro/db:/db \
  -v /mnt/user/appdata/encoderpro/logs:/logs \
  --restart unless-stopped \
  encoderpro-intel:latest dashboard

# Access dashboard at http://your-server:5000
```

---

## Configuration

### Main Config File (config.yaml)

```yaml
# Directory paths
movies_dir: /movies          # Source movies
archive_dir: /archive        # Original file archive
work_dir: /work             # Temporary work directory

# Encoding profiles
profiles:
  default: balanced_gpu

  definitions:
    high_quality_gpu:
      encoder: nvidia_nvenc_h265
      quality: 19              # Lower = better quality
      preset: slow
      description: "Highest quality NVENC encoding"

    balanced_gpu:
      encoder: nvidia_nvenc_h265
      quality: 23
      preset: medium
      description: "Balanced quality and speed"

    fast_cpu:
      encoder: cpu_x265
      quality: 23
      preset: fast
      description: "Fast CPU encoding"

# Quality checking
quality_check:
  enabled: true
  warn_threshold: 10.0        # Warn if quality drops >10 points
  error_threshold: 20.0       # Error if quality drops >20 points
  skip_on_degradation: false  # Skip files that would degrade
  prompt_on_warning: true

# Processing options
subtitle_check:
  enabled: true               # Only process files with subtitles

batch_size: 10                # Process this many files before pausing
parallel_jobs: 1              # Number of simultaneous encodes
```

### Environment Variables

```bash
# Dashboard settings
DASHBOARD_HOST=0.0.0.0
DASHBOARD_PORT=5000
DASHBOARD_DEBUG=false

# File paths
MOVIES_DIR=/movies
ARCHIVE_DIR=/archive
WORK_DIR=/work
STATE_DB=/db/state.db
LOG_DIR=/logs
CONFIG_FILE=/config/config.yaml
REENCODE_SCRIPT=/app/reencode.py
```

---

## Usage

### Web Dashboard

1. **Access the dashboard** at `http://localhost:5000`
2. **Scan your library** - Click "📂 Scan Library" to populate database
3. **Configure settings** - Set encoding profile and quality thresholds
4. **Select files** - Choose specific movies to re-encode, or process all
5. **Start encoding** - Click "▶️ Start Processing"
6. **Monitor progress** - View real-time stats, activity, and logs

### Command Line

```bash
# Scan library only (no encoding)
python reencode.py -c config.yaml --scan-only

# Process with default profile
python reencode.py -c config.yaml

# Use specific profile
python reencode.py -c config.yaml --profile high_quality_gpu

# Check quality only
python reencode.py -c config.yaml --check-quality

# Show statistics
python reencode.py -c config.yaml --stats

# Reset specific files
python reencode.py -c config.yaml --reset-file /movies/movie.mkv
```

---

## Encoding Profiles

### Pre-Configured Profiles

| Profile | Encoder | CRF | Use Case |
|---------|---------|-----|----------|
| **quality_gpu** | NVENC H.265 | 19 | Highest quality, larger files |
| **balanced_gpu** | NVENC H.265 | 23 | Balanced quality/size (default) |
| **space_saver** | NVENC H.265 | 28 | Maximum compression |
| **quality_cpu** | x265 | 19 | CPU-based high quality |
| **balanced_cpu** | x265 | 23 | CPU-based balanced |
| **intel_qsv** | Intel QSV | 23 | Intel Quick Sync |

### Custom Profiles

Create custom profiles by editing `config.yaml`:

```yaml
profiles:
  definitions:
    my_profile:
      encoder: nvidia_nvenc_h265
      quality: 21
      preset: medium
      description: "My custom settings"
```

---

## Quality Checking

encoderPro analyzes source files before encoding to prevent quality degradation.

### Quality Score (0-100)

Based on **bits per pixel (bpp)**:
- **95-100**: Near lossless (>0.5 bpp)
- **85-95**: Excellent (0.3-0.5 bpp)
- **70-85**: Good (0.2-0.3 bpp)
- **50-70**: Acceptable (0.1-0.2 bpp)
- **<50**: Poor (<0.1 bpp)

### Degradation Thresholds

- **Warn Threshold (10)**: Shows warning if quality drops >10 points
- **Error Threshold (20)**: Stops encoding if quality drops >20 points
- **Skip on Degradation**: Automatically skip files that would degrade

### Example

```
Source: 4K BluRay Remux
- Bitrate: 40 Mbps
- Resolution: 3840x2160
- Quality Score: 92 (Excellent)

Target: CRF 23 NVENC
- Estimated Bitrate: 12 Mbps
- Quality Score: 78 (Good)
- Degradation: -14 points ⚠️ WARNING

Action: Encode with warning, or skip if skip_on_degradation=true
```

---

## Database Management

### Database Tools

```bash
# Initialize new database
python init_database.py

# View database statistics
python dbmanage.py stats

# Reset file states
python dbmanage.py reset --state completed

# Export to CSV
python dbmanage.py export files.csv

# Cleanup orphaned entries
python dbmanage.py cleanup
```

### Database Schema

**Files Table:**
- `id`: Primary key
- `filepath`: Full path to file
- `relative_path`: Display path
- `state`: pending, processing, completed, failed, skipped
- `has_subtitles`: Subtitle detection result
- `original_size`: Original file size
- `encoded_size`: Encoded file size
- `error_message`: Failure/skip reason
- `profile_name`: Encoding profile used
- `encoder_used`: Actual encoder used
- `fps`: Encoding speed
- Timestamps: created_at, updated_at, started_at, completed_at

---

## API Reference

See [DASHBOARD-API.md](DASHBOARD-API.md) for complete API documentation.

### Key Endpoints

- `GET /api/stats` - Processing statistics
- `GET /api/files` - File list with filtering
- `GET /api/profiles` - Available encoding profiles
- `POST /api/jobs/start` - Start encoding
- `POST /api/jobs/stop` - Stop encoding
- `POST /api/jobs/reencode-selected` - Queue specific files
- `GET /api/health` - System health check

---

## Deployment

See **[UNRAID-DEPLOYMENT.md](UNRAID-DEPLOYMENT.md)** for complete deployment instructions including:
- Step-by-step setup for NVIDIA, Intel Arc, and CPU
- Directory permissions configuration
- Unraid template configuration
- Performance tuning guides
- Troubleshooting common issues

---

## Troubleshooting

### Common Issues

**"No files found in database"**
- Run library scan: Click "📂 Scan Library" in dashboard
- Or: `python reencode.py -c config.yaml --scan-only`

**"Reencode script not found"**
- Check `REENCODE_SCRIPT` environment variable
- Ensure `/app/reencode.py` exists in container

**"Config file not found"**
- Mount config directory: `-v /path/to/config:/config`
- Set `CONFIG_FILE=/config/config.yaml`

**"PyYAML not installed"**
- Install: `pip install pyyaml`
- Or rebuild Docker image

**"GPU not detected"**
- NVIDIA: Install nvidia-docker2, use `--gpus all`
- Intel: Use `--device=/dev/dri:/dev/dri`
- Check: `ffmpeg -encoders | grep nvenc` or `grep qsv`

**Dashboard shows errors**
- Check logs: `docker logs encoderpro`
- Check browser console (F12)
- Verify paths and permissions

### Debug Mode

```bash
# Enable debug logging
export DASHBOARD_DEBUG=true
python dashboard.py

# Or in Docker
docker run -e DASHBOARD_DEBUG=true ...
```

---

## Architecture

### Components

- **reencode.py** - Core encoding engine
- **dashboard.py** - Flask web server
- **quality_checker.py** - Quality analysis module
- **templates/** - HTML templates
- **static/** - CSS/JS assets

### Workflow

1. **Scan** - Discover media files, detect subtitles
2. **Analyze** - Check quality, calculate scores
3. **Queue** - Add to database as "pending"
4. **Process** - Encode files in order
5. **Verify** - Check output quality
6. **Archive** - Move original to archive
7. **Complete** - Update database, log results

---

## Contributing

Contributions welcome! See [CONTRIBUTING.md](CONTRIBUTING.md).

---

## Documentation

- **[UNRAID-DEPLOYMENT.md](UNRAID-DEPLOYMENT.md)** - Complete deployment guide (START HERE)
- **[DASHBOARD-API.md](DASHBOARD-API.md)** - Complete API reference
- **[DASHBOARD-GUIDE.md](DASHBOARD-GUIDE.md)** - Web dashboard user guide
- **[QUALITY-GUIDE.md](QUALITY-GUIDE.md)** - Quality analysis guide
- **[QUALITY-CHECKING.md](QUALITY-CHECKING.md)** - Quality checking technical details
- **[FEATURE-SUMMARY.md](FEATURE-SUMMARY.md)** - New feature documentation
- **[SECURITY-FIXES.md](SECURITY-FIXES.md)** - Security improvements
- **[STUCK-PROCESSING-FIX.md](STUCK-PROCESSING-FIX.md)** - Stuck file handling

---

## License

MIT License - See [LICENSE](LICENSE) for details.

---

## Support

- **Issues**: https://github.com/yourusername/encoderPro/issues
- **Documentation**: https://encoderpro.readthedocs.io
- **Discord**: https://discord.gg/encoderpro

---

## Version

**3.2.0** - Security hardened, CSRF protection, stuck file handling, direct Docker commands