feat: AI face enhancement with GFPGAN and CodeFormer (#61)

* feat(shared): add enhance-faces tool definition and i18n strings

* feat(ai): add face enhancement script with GFPGAN and CodeFormer support

Detects faces via MediaPipe dual-model approach, then enhances using
GFPGAN (proven) or CodeFormer (via codeformer-pip) with auto fallback.
Supports strength-based alpha blending with original image.

* feat(ai): add TypeScript bridge for face enhancement

* feat(api): add enhance-faces route with GFPGAN/CodeFormer support

* feat(web): add enhance-faces settings component and register in tool registry

* feat(docker): add CodeFormer dependency and model download

- Add codeformer-pip to both CPU and GPU requirements
- Download CodeFormer model (~375MB) at Docker build time
- Add CodeFormer to smoke test verification

* fix(enhance-faces): address code review findings

- Skip alpha blend for CodeFormer (strength already applied via fidelity weight)
- Hide "only enhance main face" checkbox when Best (CodeFormer) is selected
- Fix sensitivity slider labels (swap More/Fewer faces to match actual behavior)
- Register EnhanceFacesControls in pipeline step settings
- Remove model names from user-facing descriptions

* fix(enhance-faces): fix CodeFormer integration and Docker setup

- Add codeformer-pip install to Dockerfile with --no-deps to avoid numpy 2.x conflict
- Re-pin numpy==1.26.4 after codeformer-pip install
- Pin codeformer-pip==0.0.4 in requirements files
- Broaden auto-mode fallback to catch any Exception from CodeFormer

---------

Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
This commit is contained in:
stirling-image
2026-04-13 21:56:59 +08:00
committed by GitHub
co-authored by stirling-image
parent 9ddeac92b6
commit 8071fe61c5
14 changed files with 780 additions and 0 deletions
+6
View File
@@ -137,6 +137,12 @@ RUN if [ "$TARGETARCH" = "amd64" ]; then \
/opt/venv/bin/pip install mediapipe==0.10.18 \
; fi
# CodeFormer face enhancement (install with --no-deps to avoid numpy 2.x conflict)
RUN /opt/venv/bin/pip install --no-deps codeformer-pip==0.0.4 lpips
# Re-pin numpy to 1.26.4 in case any transitive dep upgraded it
RUN /opt/venv/bin/pip install numpy==1.26.4
# Pre-download and verify all ML models
# Note: on amd64, paddlepaddle-gpu can't import without the CUDA driver (only
# available at runtime). The download script gracefully skips PaddleOCR model
+31
View File
@@ -32,6 +32,13 @@ GFPGAN_MODEL_URL = (
GFPGAN_MODEL_PATH = os.path.join(GFPGAN_MODEL_DIR, "GFPGANv1.3.pth")
GFPGAN_MIN_SIZE = 300_000_000 # ~332 MB
CODEFORMER_MODEL_DIR = "/opt/models/codeformer"
CODEFORMER_MODEL_URL = (
"https://github.com/sczhou/CodeFormer/releases/download/v0.1.0/codeformer.pth"
)
CODEFORMER_MODEL_PATH = os.path.join(CODEFORMER_MODEL_DIR, "codeformer.pth")
CODEFORMER_MIN_SIZE = 350_000_000 # ~375 MB
DDCOLOR_MODEL_DIR = "/opt/models/ddcolor"
DDCOLOR_MODEL_URL = (
"https://huggingface.co/piddnad/DDColor-models/resolve/main/ddcolor_paper_tiny.pth"
@@ -167,6 +174,20 @@ def download_gfpgan_model():
print(f" GFPGANv1.3.pth downloaded ({size / 1_000_000:.1f} MB)\n")
def download_codeformer_model():
"""Download codeformer.pth pretrained weights for face enhancement."""
print("=== Downloading CodeFormer model ===")
os.makedirs(CODEFORMER_MODEL_DIR, exist_ok=True)
print(f" Downloading from {CODEFORMER_MODEL_URL}...")
urllib.request.urlretrieve(CODEFORMER_MODEL_URL, CODEFORMER_MODEL_PATH)
size = os.path.getsize(CODEFORMER_MODEL_PATH)
assert size > CODEFORMER_MIN_SIZE, (
f"CodeFormer model too small: {size} bytes (expected > {CODEFORMER_MIN_SIZE})"
)
print(f" codeformer.pth downloaded ({size / 1_000_000:.1f} MB)\n")
def download_ddcolor_model():
"""Download pre-exported DDColor ONNX model for AI photo colorization.
@@ -319,6 +340,15 @@ def smoke_test():
)
print(" GFPGAN model file verified")
# CodeFormer model file must exist
assert os.path.exists(CODEFORMER_MODEL_PATH), (
f"CodeFormer model missing: {CODEFORMER_MODEL_PATH}"
)
assert os.path.getsize(CODEFORMER_MODEL_PATH) > CODEFORMER_MIN_SIZE, (
"CodeFormer model file is too small"
)
print(" CodeFormer model file verified")
# DDColor ONNX model must exist
assert os.path.exists(DDCOLOR_ONNX_PATH), (
f"DDColor model missing: {DDCOLOR_ONNX_PATH}"
@@ -360,6 +390,7 @@ def main():
download_rembg_models()
download_realesrgan_model()
download_gfpgan_model()
download_codeformer_model()
download_ddcolor_model()
download_paddleocr_models()
download_paddleocr_vl_model()