initial comment
This commit is contained in:
414
README.md
Normal file
414
README.md
Normal file
@@ -0,0 +1,414 @@
|
||||
# encoderPro
|
||||
|
||||
**GPU-Accelerated Media Re-Encoding with Web Dashboard**
|
||||
|
||||
Modern, intelligent video encoding system with quality checking, state tracking, and a beautiful web interface for monitoring and control.
|
||||
|
||||
---
|
||||
|
||||
## Features
|
||||
|
||||
### Core Encoding
|
||||
- **Smart Quality Detection** - Analyzes source quality and warns before degradation
|
||||
- **GPU Acceleration** - NVIDIA NVENC, Intel QSV, or CPU encoding
|
||||
- **State Tracking** - SQLite database tracks all processed files
|
||||
- **Subtitle Detection** - Only processes files with subtitle streams
|
||||
- **Resume Capability** - Interrupt and resume without losing progress
|
||||
- **Profile-Based Encoding** - Pre-configured quality profiles for different use cases
|
||||
|
||||
### Web Dashboard
|
||||
- **Real-Time Monitoring** - Live stats, progress, and activity feed
|
||||
- **Job Control** - Start, stop, and manage encoding jobs
|
||||
- **File Management** - Browse files, select specific movies to re-encode
|
||||
- **Profile Selection** - Choose encoding profiles from the UI
|
||||
- **Quality Analysis** - View file details, quality scores, and savings
|
||||
- **System Health** - Monitor GPU/CPU usage, disk space, encoder status
|
||||
|
||||
### Quality Features
|
||||
- **Pre-Encode Analysis** - Calculate quality scores before encoding
|
||||
- **Degradation Detection** - Warn if encoding will lower quality
|
||||
- **HDR Detection** - Identify HDR content for special handling
|
||||
- **Bitrate Analysis** - Calculate bits-per-pixel for quality assessment
|
||||
- **Skip Protection** - Optionally skip files that would degrade
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
**For complete setup instructions, see [UNRAID-DEPLOYMENT.md](UNRAID-DEPLOYMENT.md)**
|
||||
|
||||
The deployment guide includes:
|
||||
- NVIDIA GPU setup (RTX series)
|
||||
- Intel Arc GPU setup (A-Series, integrated graphics)
|
||||
- CPU-only setup
|
||||
- Directory permissions (critical!)
|
||||
- Troubleshooting common issues
|
||||
- Performance tuning
|
||||
|
||||
### Quick Example (Intel Arc)
|
||||
|
||||
```bash
|
||||
# Build image
|
||||
docker build -f Dockerfile.intel -t encoderpro-intel:latest .
|
||||
|
||||
# Fix permissions (REQUIRED!)
|
||||
chown -R 1000:1000 /mnt/user/appdata/encoderpro/{db,logs}
|
||||
chown -R 1000:1000 /mnt/user/temp/encoderpro-work
|
||||
chown -R 1000:1000 /mnt/user/archive/movies
|
||||
|
||||
# Run dashboard
|
||||
docker run -d \
|
||||
--name encoderpro-dashboard-intel \
|
||||
--device=/dev/dri:/dev/dri \
|
||||
-e GPU_TYPE=intel \
|
||||
-p 5000:5000 \
|
||||
-v /mnt/user/movies:/movies \
|
||||
-v /mnt/user/archive/movies:/archive \
|
||||
-v /mnt/user/appdata/encoderpro/config.yaml:/config/config.yaml:ro \
|
||||
-v /mnt/user/appdata/encoderpro/db:/db \
|
||||
-v /mnt/user/appdata/encoderpro/logs:/logs \
|
||||
--restart unless-stopped \
|
||||
encoderpro-intel:latest dashboard
|
||||
|
||||
# Access dashboard at http://your-server:5000
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
### Main Config File (config.yaml)
|
||||
|
||||
```yaml
|
||||
# Directory paths
|
||||
movies_dir: /movies # Source movies
|
||||
archive_dir: /archive # Original file archive
|
||||
work_dir: /work # Temporary work directory
|
||||
|
||||
# Encoding profiles
|
||||
profiles:
|
||||
default: balanced_gpu
|
||||
|
||||
definitions:
|
||||
high_quality_gpu:
|
||||
encoder: nvidia_nvenc_h265
|
||||
quality: 19 # Lower = better quality
|
||||
preset: slow
|
||||
description: "Highest quality NVENC encoding"
|
||||
|
||||
balanced_gpu:
|
||||
encoder: nvidia_nvenc_h265
|
||||
quality: 23
|
||||
preset: medium
|
||||
description: "Balanced quality and speed"
|
||||
|
||||
fast_cpu:
|
||||
encoder: cpu_x265
|
||||
quality: 23
|
||||
preset: fast
|
||||
description: "Fast CPU encoding"
|
||||
|
||||
# Quality checking
|
||||
quality_check:
|
||||
enabled: true
|
||||
warn_threshold: 10.0 # Warn if quality drops >10 points
|
||||
error_threshold: 20.0 # Error if quality drops >20 points
|
||||
skip_on_degradation: false # Skip files that would degrade
|
||||
prompt_on_warning: true
|
||||
|
||||
# Processing options
|
||||
subtitle_check:
|
||||
enabled: true # Only process files with subtitles
|
||||
|
||||
batch_size: 10 # Process this many files before pausing
|
||||
parallel_jobs: 1 # Number of simultaneous encodes
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
```bash
|
||||
# Dashboard settings
|
||||
DASHBOARD_HOST=0.0.0.0
|
||||
DASHBOARD_PORT=5000
|
||||
DASHBOARD_DEBUG=false
|
||||
|
||||
# File paths
|
||||
MOVIES_DIR=/movies
|
||||
ARCHIVE_DIR=/archive
|
||||
WORK_DIR=/work
|
||||
STATE_DB=/db/state.db
|
||||
LOG_DIR=/logs
|
||||
CONFIG_FILE=/config/config.yaml
|
||||
REENCODE_SCRIPT=/app/reencode.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Usage
|
||||
|
||||
### Web Dashboard
|
||||
|
||||
1. **Access the dashboard** at `http://localhost:5000`
|
||||
2. **Scan your library** - Click "📂 Scan Library" to populate database
|
||||
3. **Configure settings** - Set encoding profile and quality thresholds
|
||||
4. **Select files** - Choose specific movies to re-encode, or process all
|
||||
5. **Start encoding** - Click "▶️ Start Processing"
|
||||
6. **Monitor progress** - View real-time stats, activity, and logs
|
||||
|
||||
### Command Line
|
||||
|
||||
```bash
|
||||
# Scan library only (no encoding)
|
||||
python reencode.py -c config.yaml --scan-only
|
||||
|
||||
# Process with default profile
|
||||
python reencode.py -c config.yaml
|
||||
|
||||
# Use specific profile
|
||||
python reencode.py -c config.yaml --profile high_quality_gpu
|
||||
|
||||
# Check quality only
|
||||
python reencode.py -c config.yaml --check-quality
|
||||
|
||||
# Show statistics
|
||||
python reencode.py -c config.yaml --stats
|
||||
|
||||
# Reset specific files
|
||||
python reencode.py -c config.yaml --reset-file /movies/movie.mkv
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Encoding Profiles
|
||||
|
||||
### Pre-Configured Profiles
|
||||
|
||||
| Profile | Encoder | CRF | Use Case |
|
||||
|---------|---------|-----|----------|
|
||||
| **quality_gpu** | NVENC H.265 | 19 | Highest quality, larger files |
|
||||
| **balanced_gpu** | NVENC H.265 | 23 | Balanced quality/size (default) |
|
||||
| **space_saver** | NVENC H.265 | 28 | Maximum compression |
|
||||
| **quality_cpu** | x265 | 19 | CPU-based high quality |
|
||||
| **balanced_cpu** | x265 | 23 | CPU-based balanced |
|
||||
| **intel_qsv** | Intel QSV | 23 | Intel Quick Sync |
|
||||
|
||||
### Custom Profiles
|
||||
|
||||
Create custom profiles by editing `config.yaml`:
|
||||
|
||||
```yaml
|
||||
profiles:
|
||||
definitions:
|
||||
my_profile:
|
||||
encoder: nvidia_nvenc_h265
|
||||
quality: 21
|
||||
preset: medium
|
||||
description: "My custom settings"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quality Checking
|
||||
|
||||
encoderPro analyzes source files before encoding to prevent quality degradation.
|
||||
|
||||
### Quality Score (0-100)
|
||||
|
||||
Based on **bits per pixel (bpp)**:
|
||||
- **95-100**: Near lossless (>0.5 bpp)
|
||||
- **85-95**: Excellent (0.3-0.5 bpp)
|
||||
- **70-85**: Good (0.2-0.3 bpp)
|
||||
- **50-70**: Acceptable (0.1-0.2 bpp)
|
||||
- **<50**: Poor (<0.1 bpp)
|
||||
|
||||
### Degradation Thresholds
|
||||
|
||||
- **Warn Threshold (10)**: Shows warning if quality drops >10 points
|
||||
- **Error Threshold (20)**: Stops encoding if quality drops >20 points
|
||||
- **Skip on Degradation**: Automatically skip files that would degrade
|
||||
|
||||
### Example
|
||||
|
||||
```
|
||||
Source: 4K BluRay Remux
|
||||
- Bitrate: 40 Mbps
|
||||
- Resolution: 3840x2160
|
||||
- Quality Score: 92 (Excellent)
|
||||
|
||||
Target: CRF 23 NVENC
|
||||
- Estimated Bitrate: 12 Mbps
|
||||
- Quality Score: 78 (Good)
|
||||
- Degradation: -14 points ⚠️ WARNING
|
||||
|
||||
Action: Encode with warning, or skip if skip_on_degradation=true
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Database Management
|
||||
|
||||
### Database Tools
|
||||
|
||||
```bash
|
||||
# Initialize new database
|
||||
python init_database.py
|
||||
|
||||
# View database statistics
|
||||
python dbmanage.py stats
|
||||
|
||||
# Reset file states
|
||||
python dbmanage.py reset --state completed
|
||||
|
||||
# Export to CSV
|
||||
python dbmanage.py export files.csv
|
||||
|
||||
# Cleanup orphaned entries
|
||||
python dbmanage.py cleanup
|
||||
```
|
||||
|
||||
### Database Schema
|
||||
|
||||
**Files Table:**
|
||||
- `id`: Primary key
|
||||
- `filepath`: Full path to file
|
||||
- `relative_path`: Display path
|
||||
- `state`: pending, processing, completed, failed, skipped
|
||||
- `has_subtitles`: Subtitle detection result
|
||||
- `original_size`: Original file size
|
||||
- `encoded_size`: Encoded file size
|
||||
- `error_message`: Failure/skip reason
|
||||
- `profile_name`: Encoding profile used
|
||||
- `encoder_used`: Actual encoder used
|
||||
- `fps`: Encoding speed
|
||||
- Timestamps: created_at, updated_at, started_at, completed_at
|
||||
|
||||
---
|
||||
|
||||
## API Reference
|
||||
|
||||
See [DASHBOARD-API.md](DASHBOARD-API.md) for complete API documentation.
|
||||
|
||||
### Key Endpoints
|
||||
|
||||
- `GET /api/stats` - Processing statistics
|
||||
- `GET /api/files` - File list with filtering
|
||||
- `GET /api/profiles` - Available encoding profiles
|
||||
- `POST /api/jobs/start` - Start encoding
|
||||
- `POST /api/jobs/stop` - Stop encoding
|
||||
- `POST /api/jobs/reencode-selected` - Queue specific files
|
||||
- `GET /api/health` - System health check
|
||||
|
||||
---
|
||||
|
||||
## Deployment
|
||||
|
||||
See **[UNRAID-DEPLOYMENT.md](UNRAID-DEPLOYMENT.md)** for complete deployment instructions including:
|
||||
- Step-by-step setup for NVIDIA, Intel Arc, and CPU
|
||||
- Directory permissions configuration
|
||||
- Unraid template configuration
|
||||
- Performance tuning guides
|
||||
- Troubleshooting common issues
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
**"No files found in database"**
|
||||
- Run library scan: Click "📂 Scan Library" in dashboard
|
||||
- Or: `python reencode.py -c config.yaml --scan-only`
|
||||
|
||||
**"Reencode script not found"**
|
||||
- Check `REENCODE_SCRIPT` environment variable
|
||||
- Ensure `/app/reencode.py` exists in container
|
||||
|
||||
**"Config file not found"**
|
||||
- Mount config directory: `-v /path/to/config:/config`
|
||||
- Set `CONFIG_FILE=/config/config.yaml`
|
||||
|
||||
**"PyYAML not installed"**
|
||||
- Install: `pip install pyyaml`
|
||||
- Or rebuild Docker image
|
||||
|
||||
**"GPU not detected"**
|
||||
- NVIDIA: Install nvidia-docker2, use `--gpus all`
|
||||
- Intel: Use `--device=/dev/dri:/dev/dri`
|
||||
- Check: `ffmpeg -encoders | grep nvenc` or `grep qsv`
|
||||
|
||||
**Dashboard shows errors**
|
||||
- Check logs: `docker logs encoderpro`
|
||||
- Check browser console (F12)
|
||||
- Verify paths and permissions
|
||||
|
||||
### Debug Mode
|
||||
|
||||
```bash
|
||||
# Enable debug logging
|
||||
export DASHBOARD_DEBUG=true
|
||||
python dashboard.py
|
||||
|
||||
# Or in Docker
|
||||
docker run -e DASHBOARD_DEBUG=true ...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
### Components
|
||||
|
||||
- **reencode.py** - Core encoding engine
|
||||
- **dashboard.py** - Flask web server
|
||||
- **quality_checker.py** - Quality analysis module
|
||||
- **templates/** - HTML templates
|
||||
- **static/** - CSS/JS assets
|
||||
|
||||
### Workflow
|
||||
|
||||
1. **Scan** - Discover media files, detect subtitles
|
||||
2. **Analyze** - Check quality, calculate scores
|
||||
3. **Queue** - Add to database as "pending"
|
||||
4. **Process** - Encode files in order
|
||||
5. **Verify** - Check output quality
|
||||
6. **Archive** - Move original to archive
|
||||
7. **Complete** - Update database, log results
|
||||
|
||||
---
|
||||
|
||||
## Contributing
|
||||
|
||||
Contributions welcome! See [CONTRIBUTING.md](CONTRIBUTING.md).
|
||||
|
||||
---
|
||||
|
||||
## Documentation
|
||||
|
||||
- **[UNRAID-DEPLOYMENT.md](UNRAID-DEPLOYMENT.md)** - Complete deployment guide (START HERE)
|
||||
- **[DASHBOARD-API.md](DASHBOARD-API.md)** - Complete API reference
|
||||
- **[DASHBOARD-GUIDE.md](DASHBOARD-GUIDE.md)** - Web dashboard user guide
|
||||
- **[QUALITY-GUIDE.md](QUALITY-GUIDE.md)** - Quality analysis guide
|
||||
- **[QUALITY-CHECKING.md](QUALITY-CHECKING.md)** - Quality checking technical details
|
||||
- **[FEATURE-SUMMARY.md](FEATURE-SUMMARY.md)** - New feature documentation
|
||||
- **[SECURITY-FIXES.md](SECURITY-FIXES.md)** - Security improvements
|
||||
- **[STUCK-PROCESSING-FIX.md](STUCK-PROCESSING-FIX.md)** - Stuck file handling
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
MIT License - See [LICENSE](LICENSE) for details.
|
||||
|
||||
---
|
||||
|
||||
## Support
|
||||
|
||||
- **Issues**: https://github.com/yourusername/encoderPro/issues
|
||||
- **Documentation**: https://encoderpro.readthedocs.io
|
||||
- **Discord**: https://discord.gg/encoderpro
|
||||
|
||||
---
|
||||
|
||||
## Version
|
||||
|
||||
**3.2.0** - Security hardened, CSRF protection, stuck file handling, direct Docker commands
|
||||
Reference in New Issue
Block a user