size

update branch

.python-version

@@ -0,0 +1 @@
+3.10.0

README.md

@@ -9,37 +9,94 @@
 </p>
 
 <p align="center">
-  <img src="media/demo.gif" alt="Demo GIF">
-  <img src="media/avgpcperformancedemo.gif" alt="Performance Demo GIF">
+  <img src="media/demo.gif" alt="Demo GIF" width="800">
 </p>
 
-
 ##  Disclaimer
 
-This software is intended as a productive contribution to the AI-generated media industry. It aims to assist artists with tasks like animating custom characters or using them as models for clothing, etc.
+This deepfake software is designed to be a productive tool for the AI-generated media industry. It can assist artists in animating custom characters, creating engaging content, and even using models for clothing design.
 
-We are aware of the potential for unethical applications and are committed to preventative measures. A built-in check prevents the program from processing inappropriate media (nudity, graphic content, sensitive material like war footage, etc.). We will continue to develop this project responsibly, adhering to law and ethics. We may shut down the project or add watermarks if legally required.
+We are aware of the potential for unethical applications and are committed to preventative measures. A built-in check prevents the program from processing inappropriate media (nudity, graphic content, sensitive material like war footage, etc.). We will continue to develop this project responsibly, adhering to the law and ethics. We may shut down the project or add watermarks if legally required.
+
+- Ethical Use: Users are expected to use this software responsibly and legally. If using a real person's face, obtain their consent and clearly label any output as a deepfake when sharing online.
+
+- Content Restrictions: The software includes built-in checks to prevent processing inappropriate media, such as nudity, graphic content, or sensitive material.
+
+- Legal Compliance: We adhere to all relevant laws and ethical guidelines. If legally required, we may shut down the project or add watermarks to the output.
+
+- User Responsibility: We are not responsible for end-user actions. Users must ensure their use of the software aligns with ethical standards and legal requirements.
+
+By using this software, you agree to these terms and commit to using it in a manner that respects the rights and dignity of others.
 
 Users are expected to use this software responsibly and legally. If using a real person's face, obtain their consent and clearly label any output as a deepfake when sharing online. We are not responsible for end-user actions.
 
 
-## Quick Start - Download Prebuilt
-<div style="margin: 28px 0;">
-  <div style="margin-bottom: 20px;">
-    <a href="https://hacksider.gumroad.com/l/vccdmm" target="_blank">
-      <img src="https://github.com/user-attachments/assets/c702bb7d-d9c0-466a-9ad2-02849294e540" alt="Download Button 1" style="width: 280px; display: block;">
-    </a>
-  </div>
-  <div>
-    <a href="https://krshh.gumroad.com/l/Deep-Live-Cam-Mac" target="_blank">
-      <img src="https://github.com/user-attachments/assets/9a302750-2d54-457d-bdc8-6ed7c6af0e1a" alt="Download Button 2" style="width: 280px; display: block;">
-    </a>
-  </div>
-</div>
+## Quick Start - Pre-built (Windows / Nvidia)
+
+  <a href="https://hacksider.gumroad.com/l/vccdmm"> <img src="https://github.com/user-attachments/assets/7d993b32-e3e8-4cd3-bbfb-a549152ebdd5" width="285" height="77" />
+
+##### This is the fastest build you can get if you have a discrete NVIDIA GPU.
+
+## Quick Start - Pre-built (Mac / Silicon)
+    
+  <a href="https://krshh.gumroad.com/l/Deep-Live-Cam-Mac"> <img src="https://github.com/user-attachments/assets/d5d913b5-a7de-4609-96b9-979a5749a703" width="285" height="77" />
+ 
+###### These Pre-builts are perfect for non-technical users or those who don’t have time to, or can't manually install all the requirements. Just a heads-up: this is an open-source project, so you can also install it manually.
+
+## TLDR; Live Deepfake in just 3 Clicks
+![easysteps](https://github.com/user-attachments/assets/af825228-852c-411b-b787-ffd9aac72fc6)
+1. Select a face
+2. Select which camera to use
+3. Press live!
+
+## Features & Uses - Everything is real-time
+
+### Mouth Mask
+
+**Retain your original mouth for accurate movement using Mouth Mask**
+
+<p align="center">
+  <img src="media/ludwig.gif" alt="resizable-gif">
+</p>
+
+### Face Mapping
+
+**Use different faces on multiple subjects simultaneously**
+
+<p align="center">
+  <img src="media/streamers.gif" alt="face_mapping_source">
+</p>
+
+### Your Movie, Your Face
+
+**Watch movies with any face in real-time**
+
+<p align="center">
+  <img src="media/movie.gif" alt="movie">
+</p>
+
+### Live Show
+
+**Run Live shows and performances**
+
+<p align="center">
+  <img src="media/live_show.gif" alt="show">
+</p>
+
+### Memes
+
+**Create Your most viral meme yet**
+
+<p align="center">
+  <img src="media/meme.gif" alt="show" width="450"> 
+  <br>
+  <sub>Created using Many Faces feature in Deep-Live-Cam</sub>
+</p>
 
 
 ## Installation (Manual)
-**Please be aware that the installation needs technical skills and is not for beginners, consider downloading the prebuilt.**
+
+**Please be aware that the installation requires technical skills and is not for beginners. Consider downloading the prebuilt version.**
 
 <details>
 <summary>Click to see the process</summary>
@@ -53,19 +110,19 @@ This is more likely to work on your computer but will be slower as it utilizes t
 -   Python (3.10 recommended)
 -   pip
 -   git
-- [ffmpeg](https://www.youtube.com/watch?v=OlNWCpFdVMA) 
+-   [ffmpeg](https://www.youtube.com/watch?v=OlNWCpFdVMA) - ```iex (irm ffmpeg.tc.ht)```
 -   [Visual Studio 2022 Runtimes (Windows)](https://visualstudio.microsoft.com/visual-cpp-build-tools/)
 
-**2. Clone Repository**
+**2. Clone the Repository**
 
 ```bash
 https://github.com/hacksider/Deep-Live-Cam.git
 ```
 
-**3. Download Models**
+**3. Download the Models**
 
 1. [GFPGANv1.4](https://huggingface.co/hacksider/deep-live-cam/resolve/main/GFPGANv1.4.pth)
-2. [inswapper_128_fp16.onnx](https://huggingface.co/hacksider/deep-live-cam/resolve/main/inswapper_128.onnx) (Note: Use this [replacement version](https://github.com/facefusion/facefusion-assets/releases/download/models/inswapper_128.onnx) if you encounter issues)
+2. [inswapper\_128\_fp16.onnx](https://huggingface.co/hacksider/deep-live-cam/resolve/main/inswapper_128_fp16.onnx)
 
 Place these files in the "**models**" folder.
 
@@ -85,18 +142,20 @@ brew install python-tk@3.10
 
 **Run:** If you don't have a GPU, you can run Deep-Live-Cam using `python run.py`. Note that initial execution will download models (~300MB).
 
-
 ### GPU Acceleration
 
 **CUDA Execution Provider (Nvidia)**
 
 1. Install [CUDA Toolkit 11.8](https://developer.nvidia.com/cuda-11-8-0-download-archive) or [CUDA Toolkit 12.1.1](https://developer.nvidia.com/cuda-12-1-1-download-archive)
 2. Install dependencies:
+
 ```bash
 pip uninstall onnxruntime onnxruntime-gpu
 pip install onnxruntime-gpu==1.16.3
 ```
+
 3. Usage:
+
 ```bash
 python run.py --execution-provider cuda
 ```
@@ -104,11 +163,14 @@ python run.py --execution-provider cuda
 **CoreML Execution Provider (Apple Silicon)**
 
 1. Install dependencies:
+
 ```bash
 pip uninstall onnxruntime onnxruntime-silicon
 pip install onnxruntime-silicon==1.13.1
 ```
+
 2. Usage:
+
 ```bash
 python run.py --execution-provider coreml
 ```
@@ -116,11 +178,14 @@ python run.py --execution-provider coreml
 **CoreML Execution Provider (Apple Legacy)**
 
 1. Install dependencies:
+
 ```bash
 pip uninstall onnxruntime onnxruntime-coreml
 pip install onnxruntime-coreml==1.13.1
 ```
+
 2. Usage:
+
 ```bash
 python run.py --execution-provider coreml
 ```
@@ -128,11 +193,14 @@ python run.py --execution-provider coreml
 **DirectML Execution Provider (Windows)**
 
 1. Install dependencies:
+
 ```bash
 pip uninstall onnxruntime onnxruntime-directml
 pip install onnxruntime-directml==1.15.1
 ```
+
 2. Usage:
+
 ```bash
 python run.py --execution-provider directml
 ```
@@ -140,18 +208,20 @@ python run.py --execution-provider directml
 **OpenVINO™ Execution Provider (Intel)**
 
 1. Install dependencies:
+
 ```bash
 pip uninstall onnxruntime onnxruntime-openvino
 pip install onnxruntime-openvino==1.15.0
 ```
+
 2. Usage:
+
 ```bash
 python run.py --execution-provider openvino
 ```
 
 </details>
 
-
 ## Usage
 
 **1. Image/Video Mode**
@@ -170,33 +240,6 @@ python run.py --execution-provider openvino
 -   Use a screen capture tool like OBS to stream.
 -   To change the face, select a new source image.
 
-## Features - Everything is realtime
-
-### Mouth Mask
-
-**Retain your original mouth using Mouth Mask** 
-
-![resizable-gif](media/ludwig.gif)
-
-### Face Mapping
-
-**Use different faces on multiple subjects**
-
-![face_mapping_source](media/streamers.gif)
-
-### Your Movie, Your Face
-
-**Watch movies with any face in realtime**
-
-![movie](media/movie.gif)
-
-
-## Benchmarks
-
-**Nearly 0% detection!**
-
-![bench](media/deepwarebench.gif)
-
 ## Command Line Arguments (Unmaintained)
 
 ```
@@ -212,7 +255,6 @@ options:
   --many-faces                                             process every face
   --map-faces                                              map source target faces
   --mouth-mask                                             mask the mouth region
-  --nsfw-filter                                            filter the NSFW image or video
   --video-encoder {libx264,libx265,libvpx-vp9}             adjust output video encoder
   --video-quality [0-51]                                   adjust output video quality
   --live-mirror                                            the live camera display as you see it in the front-facing camera frame
@@ -225,9 +267,9 @@ options:
 
 Looking for a CLI mode? Using the -s/--source argument will make the run program in cli mode.
 
-
 ## Press
- **We are always open to criticism and ready to improve, that's why we didn't cherrypick anything.** 
+
+**We are always open to criticism and are ready to improve, that's why we didn't cherry-pick anything.**
 
  - [*"Deep-Live-Cam goes viral, allowing anyone to become a digital doppelganger"*](https://arstechnica.com/information-technology/2024/08/new-ai-tool-enables-real-time-face-swapping-on-webcams-raising-fraud-concerns/) - Ars Technica
  - [*"Thanks Deep Live Cam, shapeshifters are among us now"*](https://dataconomy.com/2024/08/15/what-is-deep-live-cam-github-deepfake/) - Dataconomy
@@ -242,25 +284,26 @@ Looking for a CLI mode? Using the -s/--source argument will make the run program
  - [*"That's Crazy, Oh God. That's Fucking Freaky Dude... That's So Wild Dude"*](https://www.youtube.com/watch?time_continue=1074&v=py4Tc-Y8BcY) - SomeOrdinaryGamers
  - [*"Alright look look look, now look chat, we can do any face we want to look like chat"*](https://www.youtube.com/live/mFsCe7AIxq8?feature=shared&t=2686) - IShowSpeed
 
-
 ## Credits
 
-- [ffmpeg](https://ffmpeg.org/): for making video related operations easy
+-   [ffmpeg](https://ffmpeg.org/): for making video-related operations easy
 -   [deepinsight](https://github.com/deepinsight): for their [insightface](https://github.com/deepinsight/insightface) project which provided a well-made library and models. Please be reminded that the [use of the model is for non-commercial research purposes only](https://github.com/deepinsight/insightface?tab=readme-ov-file#license).
 -   [havok2-htwo](https://github.com/havok2-htwo): for sharing the code for webcam
-- [GosuDRM](https://github.com/GosuDRM) : for open version of roop
+-   [GosuDRM](https://github.com/GosuDRM): for the open version of roop
 -   [pereiraroland26](https://github.com/pereiraroland26): Multiple faces support
-- [vic4key](https://github.com/vic4key) : For supporting/contributing on this project
-- [KRSHH](https://github.com/KRSHH) : For his contributions
+-   [vic4key](https://github.com/vic4key): For supporting/contributing to this project
 -   [kier007](https://github.com/kier007): for improving the user experience
+-   [qitianai](https://github.com/qitianai): for multi-lingual support
 -   and [all developers](https://github.com/hacksider/Deep-Live-Cam/graphs/contributors) behind libraries used in this project.
-- Foot Note: Please be informed that the base author of the code is [s0md3v](https://github.com/s0md3v/roop)
-- All the wonderful users who helped making this project go viral by starring the repo ❤️
+-   Footnote: Please be informed that the base author of the code is [s0md3v](https://github.com/s0md3v/roop)
+-   All the wonderful users who helped make this project go viral by starring the repo ❤️
 
 [![Stargazers](https://reporoster.com/stars/hacksider/Deep-Live-Cam)](https://github.com/hacksider/Deep-Live-Cam/stargazers)
 
 ## Contributions
+
 ![Alt](https://repobeats.axiom.co/api/embed/fec8e29c45dfdb9c5916f3a7830e1249308d20e1.svg "Repobeats analytics image")
+
 ## Stars to the Moon 🚀
 
 <a href="https://star-history.com/#hacksider/deep-live-cam&Date">
@@ -270,3 +313,5 @@ Looking for a CLI mode? Using the -s/--source argument will make the run program
    <img alt="Star History Chart" src="https://api.star-history.com/svg?repos=hacksider/deep-live-cam&type=Date" />
  </picture>
 </a>
+
+

locales/zh.json

@@ -0,0 +1,46 @@
+{
+    "Source x Target Mapper": "Source x Target Mapper",
+    "select an source image": "选择一个源图像",
+    "Preview": "预览",
+    "select an target image or video": "选择一个目标图像或视频",
+    "save image output file": "保存图像输出文件",
+    "save video output file": "保存视频输出文件",
+    "select an target image": "选择一个目标图像",
+    "source": "源",
+    "Select a target": "选择一个目标",
+    "Select a face": "选择一张脸",
+    "Keep audio": "保留音频",
+    "Face Enhancer": "面纹增强器",
+    "Many faces": "多脸",
+    "Show FPS": "显示帧率",
+    "Keep fps": "保持帧率",
+    "Keep frames": "保持帧数",
+    "Fix Blueish Cam": "修复偏蓝的摄像头",
+    "Mouth Mask": "口罩",
+    "Show Mouth Mask Box": "显示口罩盒",
+    "Start": "开始",
+    "Live": "直播",
+    "Destroy": "结束",
+    "Map faces": "识别人脸",
+    "Processing...": "处理中...",
+    "Processing succeed!": "处理成功!",
+    "Processing ignored!": "处理被忽略!",
+    "Failed to start camera": "启动相机失败",
+    "Please complete pop-up or close it.": "请先完成弹出窗口或者关闭它",
+    "Getting unique faces": "获取独特面部",
+    "Please select a source image first": "请先选择一个源图像",
+    "No faces found in target": "目标图像中没有人脸",
+    "Add": "添加",
+    "Clear": "清除",
+    "Submit": "确认",
+    "Select source image": "请选取源图像",
+    "Select target image": "请选取目标图像",
+    "Please provide mapping!": "请提供映射",
+    "Atleast 1 source with target is required!": "至少需要一个来源图像与目标图像相关！",
+    "At least 1 source with target is required!": "至少需要一个来源图像与目标图像相关！",
+    "Face could not be detected in last upload!": "最近上传的图像中没有检测到人脸!",
+    "Select Camera:": "选择摄像头",
+    "All mappings cleared!": "所有映射均已清除!",
+    "Mappings successfully submitted!": "成功提交映射！",
+    "Source x Target Mapper is already open.": "源 x 目标映射器已打开。"
+}

modules/core.py

@@ -20,6 +20,7 @@ import modules.metadata
 import modules.ui as ui
 from modules.processors.frame.core import get_frame_processors_modules
 from modules.utilities import has_image_extension, is_image, is_video, detect_fps, create_video, extract_frames, get_temp_frame_paths, restore_audio, create_temp, move_temp, clean_temp, normalize_output_path
+from modules.fake_face_handler import cleanup_fake_face
 
 if 'ROCMExecutionProvider' in modules.globals.execution_providers:
     del torch
@@ -35,15 +36,14 @@ def parse_args() -> None:
     program.add_argument('-t', '--target', help='select an target image or video', dest='target_path')
     program.add_argument('-o', '--output', help='select output file or directory', dest='output_path')
     program.add_argument('--frame-processor', help='pipeline of frame processors', dest='frame_processor', default=['face_swapper'], choices=['face_swapper', 'face_enhancer'], nargs='+')
-    program.add_argument('--keep-fps', help='keep original fps', dest='keep_fps', action='store_true', default=False)
     program.add_argument('--keep-audio', help='keep original audio', dest='keep_audio', action='store_true', default=True)
-    program.add_argument('--keep-frames', help='keep temporary frames', dest='keep_frames', action='store_true', default=False)
     program.add_argument('--many-faces', help='process every face', dest='many_faces', action='store_true', default=False)
     program.add_argument('--nsfw-filter', help='filter the NSFW image or video', dest='nsfw_filter', action='store_true', default=False)
     program.add_argument('--map-faces', help='map source target faces', dest='map_faces', action='store_true', default=False)
     program.add_argument('--mouth-mask', help='mask the mouth region', dest='mouth_mask', action='store_true', default=False)
     program.add_argument('--video-encoder', help='adjust output video encoder', dest='video_encoder', default='libx264', choices=['libx264', 'libx265', 'libvpx-vp9'])
     program.add_argument('--video-quality', help='adjust output video quality', dest='video_quality', type=int, default=18, choices=range(52), metavar='[0-51]')
+    program.add_argument('-l', '--lang', help='Ui language', default="en")
     program.add_argument('--live-mirror', help='The live camera display as you see it in the front-facing camera frame', dest='live_mirror', action='store_true', default=False)
     program.add_argument('--live-resizable', help='The live camera frame is resizable', dest='live_resizable', action='store_true', default=False)
     program.add_argument('--max-memory', help='maximum amount of RAM in GB', dest='max_memory', type=int, default=suggest_max_memory())
@@ -64,9 +64,9 @@ def parse_args() -> None:
     modules.globals.output_path = normalize_output_path(modules.globals.source_path, modules.globals.target_path, args.output_path)
     modules.globals.frame_processors = args.frame_processor
     modules.globals.headless = args.source_path or args.target_path or args.output_path
-    modules.globals.keep_fps = args.keep_fps
+    modules.globals.keep_fps = True
+    modules.globals.keep_frames = True
     modules.globals.keep_audio = args.keep_audio
-    modules.globals.keep_frames = args.keep_frames
     modules.globals.many_faces = args.many_faces
     modules.globals.mouth_mask = args.mouth_mask
     modules.globals.nsfw_filter = args.nsfw_filter
@@ -78,6 +78,7 @@ def parse_args() -> None:
     modules.globals.max_memory = args.max_memory
     modules.globals.execution_providers = decode_execution_providers(args.execution_provider)
     modules.globals.execution_threads = args.execution_threads
+    modules.globals.lang = args.lang
 
     #for ENHANCER tumbler:
     if 'face_enhancer' in args.frame_processor:
@@ -239,6 +240,7 @@ def start() -> None:
 def destroy(to_quit=True) -> None:
     if modules.globals.target_path:
         clean_temp(modules.globals.target_path)
+    cleanup_fake_face()
     if to_quit: quit()
 
 
@@ -253,5 +255,5 @@ def run() -> None:
     if modules.globals.headless:
         start()
     else:
-        window = ui.init(start, destroy)
+        window = ui.init(start, destroy, modules.globals.lang)
         window.mainloop()

modules/fake_face_handler.py

@@ -0,0 +1,120 @@
+import os
+import requests
+import tempfile
+from pathlib import Path
+import cv2
+import numpy as np
+import modules.globals
+
+def add_padding_to_face(image, padding_ratio=0.3):
+    """Add padding around the face image
+    
+    Args:
+        image: The input face image
+        padding_ratio: Amount of padding to add as a ratio of image dimensions
+        
+    Returns:
+        Padded image with background padding added
+    """
+    if image is None:
+        return None
+        
+    height, width = image.shape[:2]
+    pad_x = int(width * padding_ratio)
+    pad_y = int(height * padding_ratio)
+    
+    # Create larger image with padding
+    padded_height = height + 2 * pad_y
+    padded_width = width + 2 * pad_x
+    padded_image = np.zeros((padded_height, padded_width, 3), dtype=np.uint8)
+    
+    # Fill padded area with blurred and darkened edge pixels
+    edge_color = cv2.blur(image, (15, 15))
+    edge_color = (edge_color * 0.6).astype(np.uint8)  # Darken the padding
+    
+    # Fill the padded image with original face
+    padded_image[pad_y:pad_y+height, pad_x:pad_x+width] = image
+    
+    # Fill padding areas with edge color
+    # Top padding - repeat first row
+    top_edge = edge_color[0, :, :]
+    for i in range(pad_y):
+        padded_image[i, pad_x:pad_x+width] = top_edge
+        
+    # Bottom padding - repeat last row
+    bottom_edge = edge_color[-1, :, :]
+    for i in range(pad_y):
+        padded_image[pad_y+height+i, pad_x:pad_x+width] = bottom_edge
+        
+    # Left padding - repeat first column
+    left_edge = edge_color[:, 0, :]
+    for i in range(pad_x):
+        padded_image[pad_y:pad_y+height, i] = left_edge
+        
+    # Right padding - repeat last column
+    right_edge = edge_color[:, -1, :]
+    for i in range(pad_x):
+        padded_image[pad_y:pad_y+height, pad_x+width+i] = right_edge
+        
+    # Fill corners with nearest edge colors
+    # Top-left corner
+    padded_image[:pad_y, :pad_x] = edge_color[0, 0, :]
+    # Top-right corner
+    padded_image[:pad_y, pad_x+width:] = edge_color[0, -1, :]
+    # Bottom-left corner
+    padded_image[pad_y+height:, :pad_x] = edge_color[-1, 0, :]
+    # Bottom-right corner
+    padded_image[pad_y+height:, pad_x+width:] = edge_color[-1, -1, :]
+    
+    return padded_image
+
+def get_fake_face() -> str:
+    """Fetch a face from thispersondoesnotexist.com and save it temporarily"""
+    try:
+        # Create temp directory if it doesn't exist
+        temp_dir = Path(tempfile.gettempdir()) / "deep-live-cam"
+        temp_dir.mkdir(parents=True, exist_ok=True)
+        
+        # Generate temp file path
+        temp_file = temp_dir / "fake_face.jpg"
+        
+        # Basic headers to mimic a browser request
+        headers = {
+            'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36'
+        }
+        
+        # Fetch the image
+        response = requests.get('https://thispersondoesnotexist.com', headers=headers)
+        
+        if response.status_code == 200:
+            # Read image from response
+            image_array = np.asarray(bytearray(response.content), dtype=np.uint8)
+            image = cv2.imdecode(image_array, cv2.IMREAD_COLOR)
+            
+            # Add padding around the face
+            padded_image = add_padding_to_face(image)
+            
+            # Save the padded image
+            cv2.imwrite(str(temp_file), padded_image)
+            return str(temp_file)
+        else:
+            print(f"Failed to fetch fake face: {response.status_code}")
+            return None
+    except Exception as e:
+        print(f"Error fetching fake face: {str(e)}")
+        return None
+
+def cleanup_fake_face():
+    """Clean up the temporary fake face image"""
+    try:
+        if modules.globals.fake_face_path and os.path.exists(modules.globals.fake_face_path):
+            os.remove(modules.globals.fake_face_path)
+            modules.globals.fake_face_path = None
+    except Exception as e:
+        print(f"Error cleaning up fake face: {str(e)}")
+
+def refresh_fake_face():
+    """Refresh the fake face image"""
+    cleanup_fake_face()
+    modules.globals.fake_face_path = get_fake_face()
+    return modules.globals.fake_face_path is not None 

modules/gettext.py

@@ -0,0 +1,26 @@
+import json
+from pathlib import Path
+
+class LanguageManager:
+    def __init__(self, default_language="en"):
+        self.current_language = default_language
+        self.translations = {}
+        self.load_language(default_language)
+
+    def load_language(self, language_code) -> bool:
+        """load language file"""
+        if language_code == "en":
+            return True
+        try:
+            file_path = Path(__file__).parent.parent / f"locales/{language_code}.json"
+            with open(file_path, "r", encoding="utf-8") as file:
+                self.translations = json.load(file)
+            self.current_language = language_code
+            return True
+        except FileNotFoundError:
+            print(f"Language file not found: {language_code}")
+            return False
+
+    def _(self, key, default=None) -> str:
+        """get translate text"""
+        return self.translations.get(key, default if default else key)

modules/globals.py

@@ -21,7 +21,7 @@ keep_audio = True
 keep_frames = False
 many_faces = False
 map_faces = False
-color_correction = False  # New global variable for color correction toggle
+color_correction = False
 nsfw_filter = False
 video_encoder = None
 video_quality = None
@@ -41,3 +41,12 @@ show_mouth_mask_box = False
 mask_feather_ratio = 8
 mask_down_size = 0.50
 mask_size = 1
+mouth_mask_size = 1.0
+eyes_mask = False
+show_eyes_mask_box = False
+eyebrows_mask = False
+show_eyebrows_mask_box = False
+eyes_mask_size = 1.0
+eyebrows_mask_size = 1.0
+use_fake_face = False
+fake_face_path = None

modules/processors/frame/face_masking.py

@@ -0,0 +1,634 @@
+import cv2
+import numpy as np
+from modules.typing import Face, Frame
+import modules.globals
+
+def apply_color_transfer(source, target):
+    """
+    Apply color transfer from target to source image
+    """
+    source = cv2.cvtColor(source, cv2.COLOR_BGR2LAB).astype("float32")
+    target = cv2.cvtColor(target, cv2.COLOR_BGR2LAB).astype("float32")
+
+    source_mean, source_std = cv2.meanStdDev(source)
+    target_mean, target_std = cv2.meanStdDev(target)
+
+    # Reshape mean and std to be broadcastable
+    source_mean = source_mean.reshape(1, 1, 3)
+    source_std = source_std.reshape(1, 1, 3)
+    target_mean = target_mean.reshape(1, 1, 3)
+    target_std = target_std.reshape(1, 1, 3)
+
+    # Perform the color transfer
+    source = (source - source_mean) * (target_std / source_std) + target_mean
+
+    return cv2.cvtColor(np.clip(source, 0, 255).astype("uint8"), cv2.COLOR_LAB2BGR)
+
+def create_face_mask(face: Face, frame: Frame) -> np.ndarray:
+    mask = np.zeros(frame.shape[:2], dtype=np.uint8)
+    landmarks = face.landmark_2d_106
+    if landmarks is not None:
+        # Convert landmarks to int32
+        landmarks = landmarks.astype(np.int32)
+
+        # Extract facial features
+        right_side_face = landmarks[0:16]
+        left_side_face = landmarks[17:32]
+        right_eye = landmarks[33:42]
+        right_eye_brow = landmarks[43:51]
+        left_eye = landmarks[87:96]
+        left_eye_brow = landmarks[97:105]
+
+        # Calculate forehead extension
+        right_eyebrow_top = np.min(right_eye_brow[:, 1])
+        left_eyebrow_top = np.min(left_eye_brow[:, 1])
+        eyebrow_top = min(right_eyebrow_top, left_eyebrow_top)
+
+        face_top = np.min([right_side_face[0, 1], left_side_face[-1, 1]])
+        forehead_height = face_top - eyebrow_top
+        extended_forehead_height = int(forehead_height * 5.0)  # Extend by 50%
+
+        # Create forehead points
+        forehead_left = right_side_face[0].copy()
+        forehead_right = left_side_face[-1].copy()
+        forehead_left[1] -= extended_forehead_height
+        forehead_right[1] -= extended_forehead_height
+
+        # Combine all points to create the face outline
+        face_outline = np.vstack(
+            [
+                [forehead_left],
+                right_side_face,
+                left_side_face[::-1],  # Reverse left side to create a continuous outline
+                [forehead_right],
+            ]
+        )
+
+        # Calculate padding
+        padding = int(
+            np.linalg.norm(right_side_face[0] - left_side_face[-1]) * 0.05
+        )  # 5% of face width
+
+        # Create a slightly larger convex hull for padding
+        hull = cv2.convexHull(face_outline)
+        hull_padded = []
+        for point in hull:
+            x, y = point[0]
+            center = np.mean(face_outline, axis=0)
+            direction = np.array([x, y]) - center
+            direction = direction / np.linalg.norm(direction)
+            padded_point = np.array([x, y]) + direction * padding
+            hull_padded.append(padded_point)
+
+        hull_padded = np.array(hull_padded, dtype=np.int32)
+
+        # Fill the padded convex hull
+        cv2.fillConvexPoly(mask, hull_padded, 255)
+
+        # Smooth the mask edges
+        mask = cv2.GaussianBlur(mask, (5, 5), 3)
+
+    return mask
+
+def create_lower_mouth_mask(
+    face: Face, frame: Frame
+) -> (np.ndarray, np.ndarray, tuple, np.ndarray):
+    mask = np.zeros(frame.shape[:2], dtype=np.uint8)
+    mouth_cutout = None
+    landmarks = face.landmark_2d_106
+    if landmarks is not None:
+        #                  0  1  2  3  4  5  6  7  8  9  10 11 12 13 14 15 16 17 18 19 20
+        lower_lip_order = [
+            65,
+            66,
+            62,
+            70,
+            69,
+            18,
+            19,
+            20,
+            21,
+            22,
+            23,
+            24,
+            0,
+            8,
+            7,
+            6,
+            5,
+            4,
+            3,
+            2,
+            65,
+        ]
+        lower_lip_landmarks = landmarks[lower_lip_order].astype(
+            np.float32
+        )  # Use float for precise calculations
+
+        # Calculate the center of the landmarks
+        center = np.mean(lower_lip_landmarks, axis=0)
+
+        # Expand the landmarks outward using the mouth_mask_size
+        expansion_factor = (
+            1 + modules.globals.mask_down_size * modules.globals.mouth_mask_size
+        )  # Adjust expansion based on slider
+        expanded_landmarks = (lower_lip_landmarks - center) * expansion_factor + center
+
+        # Extend the top lip part
+        toplip_indices = [
+            20,
+            0,
+            1,
+            2,
+            3,
+            4,
+            5,
+        ]  # Indices for landmarks 2, 65, 66, 62, 70, 69, 18
+        toplip_extension = (
+            modules.globals.mask_size * modules.globals.mouth_mask_size * 0.5
+        )  # Adjust extension based on slider
+        for idx in toplip_indices:
+            direction = expanded_landmarks[idx] - center
+            direction = direction / np.linalg.norm(direction)
+            expanded_landmarks[idx] += direction * toplip_extension
+
+        # Extend the bottom part (chin area)
+        chin_indices = [
+            11,
+            12,
+            13,
+            14,
+            15,
+            16,
+        ]  # Indices for landmarks 21, 22, 23, 24, 0, 8
+        chin_extension = 2 * 0.2  # Adjust this factor to control the extension
+        for idx in chin_indices:
+            expanded_landmarks[idx][1] += (
+                expanded_landmarks[idx][1] - center[1]
+            ) * chin_extension
+
+        # Convert back to integer coordinates
+        expanded_landmarks = expanded_landmarks.astype(np.int32)
+
+        # Calculate bounding box for the expanded lower mouth
+        min_x, min_y = np.min(expanded_landmarks, axis=0)
+        max_x, max_y = np.max(expanded_landmarks, axis=0)
+
+        # Add some padding to the bounding box
+        padding = int((max_x - min_x) * 0.1)  # 10% padding
+        min_x = max(0, min_x - padding)
+        min_y = max(0, min_y - padding)
+        max_x = min(frame.shape[1], max_x + padding)
+        max_y = min(frame.shape[0], max_y + padding)
+
+        # Ensure the bounding box dimensions are valid
+        if max_x <= min_x or max_y <= min_y:
+            if (max_x - min_x) <= 1:
+                max_x = min_x + 1
+            if (max_y - min_y) <= 1:
+                max_y = min_y + 1
+
+        # Create the mask
+        mask_roi = np.zeros((max_y - min_y, max_x - min_x), dtype=np.uint8)
+        cv2.fillPoly(mask_roi, [expanded_landmarks - [min_x, min_y]], 255)
+
+        # Apply Gaussian blur to soften the mask edges
+        mask_roi = cv2.GaussianBlur(mask_roi, (15, 15), 5)
+
+        # Place the mask ROI in the full-sized mask
+        mask[min_y:max_y, min_x:max_x] = mask_roi
+
+        # Extract the masked area from the frame
+        mouth_cutout = frame[min_y:max_y, min_x:max_x].copy()
+
+        # Return the expanded lower lip polygon in original frame coordinates
+        lower_lip_polygon = expanded_landmarks
+
+    return mask, mouth_cutout, (min_x, min_y, max_x, max_y), lower_lip_polygon
+
+def create_eyes_mask(face: Face, frame: Frame) -> (np.ndarray, np.ndarray, tuple, np.ndarray):
+    mask = np.zeros(frame.shape[:2], dtype=np.uint8)
+    eyes_cutout = None
+    landmarks = face.landmark_2d_106
+    if landmarks is not None:
+        # Left eye landmarks (87-96) and right eye landmarks (33-42)
+        left_eye = landmarks[87:96]
+        right_eye = landmarks[33:42]
+        
+        # Calculate centers and dimensions for each eye
+        left_eye_center = np.mean(left_eye, axis=0).astype(np.int32)
+        right_eye_center = np.mean(right_eye, axis=0).astype(np.int32)
+        
+        # Calculate eye dimensions with size adjustment
+        def get_eye_dimensions(eye_points):
+            x_coords = eye_points[:, 0]
+            y_coords = eye_points[:, 1]
+            width = int((np.max(x_coords) - np.min(x_coords)) * (1 + modules.globals.mask_down_size * modules.globals.eyes_mask_size))
+            height = int((np.max(y_coords) - np.min(y_coords)) * (1 + modules.globals.mask_down_size * modules.globals.eyes_mask_size))
+            return width, height
+        
+        left_width, left_height = get_eye_dimensions(left_eye)
+        right_width, right_height = get_eye_dimensions(right_eye)
+        
+        # Add extra padding
+        padding = int(max(left_width, right_width) * 0.2)
+        
+        # Calculate bounding box for both eyes
+        min_x = min(left_eye_center[0] - left_width//2, right_eye_center[0] - right_width//2) - padding
+        max_x = max(left_eye_center[0] + left_width//2, right_eye_center[0] + right_width//2) + padding
+        min_y = min(left_eye_center[1] - left_height//2, right_eye_center[1] - right_height//2) - padding
+        max_y = max(left_eye_center[1] + left_height//2, right_eye_center[1] + right_height//2) + padding
+        
+        # Ensure coordinates are within frame bounds
+        min_x = max(0, min_x)
+        min_y = max(0, min_y)
+        max_x = min(frame.shape[1], max_x)
+        max_y = min(frame.shape[0], max_y)
+        
+        # Create mask for the eyes region
+        mask_roi = np.zeros((max_y - min_y, max_x - min_x), dtype=np.uint8)
+        
+        # Draw ellipses for both eyes
+        left_center = (left_eye_center[0] - min_x, left_eye_center[1] - min_y)
+        right_center = (right_eye_center[0] - min_x, right_eye_center[1] - min_y)
+        
+        # Calculate axes lengths (half of width and height)
+        left_axes = (left_width//2, left_height//2)
+        right_axes = (right_width//2, right_height//2)
+        
+        # Draw filled ellipses
+        cv2.ellipse(mask_roi, left_center, left_axes, 0, 0, 360, 255, -1)
+        cv2.ellipse(mask_roi, right_center, right_axes, 0, 0, 360, 255, -1)
+        
+        # Apply Gaussian blur to soften mask edges
+        mask_roi = cv2.GaussianBlur(mask_roi, (15, 15), 5)
+        
+        # Place the mask ROI in the full-sized mask
+        mask[min_y:max_y, min_x:max_x] = mask_roi
+        
+        # Extract the masked area from the frame
+        eyes_cutout = frame[min_y:max_y, min_x:max_x].copy()
+        
+        # Create polygon points for visualization
+        def create_ellipse_points(center, axes):
+            t = np.linspace(0, 2*np.pi, 32)
+            x = center[0] + axes[0] * np.cos(t)
+            y = center[1] + axes[1] * np.sin(t)
+            return np.column_stack((x, y)).astype(np.int32)
+        
+        # Generate points for both ellipses
+        left_points = create_ellipse_points((left_eye_center[0], left_eye_center[1]), (left_width//2, left_height//2))
+        right_points = create_ellipse_points((right_eye_center[0], right_eye_center[1]), (right_width//2, right_height//2))
+        
+        # Combine points for both eyes
+        eyes_polygon = np.vstack([left_points, right_points])
+        
+    return mask, eyes_cutout, (min_x, min_y, max_x, max_y), eyes_polygon
+
+def create_curved_eyebrow(points):
+    if len(points) >= 5:
+        # Sort points by x-coordinate
+        sorted_idx = np.argsort(points[:, 0])
+        sorted_points = points[sorted_idx]
+        
+        # Calculate dimensions
+        x_min, y_min = np.min(sorted_points, axis=0)
+        x_max, y_max = np.max(sorted_points, axis=0)
+        width = x_max - x_min
+        height = y_max - y_min
+        
+        # Create more points for smoother curve
+        num_points = 50
+        x = np.linspace(x_min, x_max, num_points)
+        
+        # Fit quadratic curve through points for more natural arch
+        coeffs = np.polyfit(sorted_points[:, 0], sorted_points[:, 1], 2)
+        y = np.polyval(coeffs, x)
+        
+        # Increased offsets to create more separation
+        top_offset = height * 0.5  # Increased from 0.3 to shift up more
+        bottom_offset = height * 0.2  # Increased from 0.1 to shift down more
+        
+        # Create smooth curves
+        top_curve = y - top_offset
+        bottom_curve = y + bottom_offset
+        
+        # Create curved endpoints with more pronounced taper
+        end_points = 5
+        start_x = np.linspace(x[0] - width * 0.15, x[0], end_points)  # Increased taper
+        end_x = np.linspace(x[-1], x[-1] + width * 0.15, end_points)  # Increased taper
+        
+        # Create tapered ends
+        start_curve = np.column_stack((
+            start_x,
+            np.linspace(bottom_curve[0], top_curve[0], end_points)
+        ))
+        end_curve = np.column_stack((
+            end_x,
+            np.linspace(bottom_curve[-1], top_curve[-1], end_points)
+        ))
+        
+        # Combine all points to form a smooth contour
+        contour_points = np.vstack([
+            start_curve,
+            np.column_stack((x, top_curve)),
+            end_curve,
+            np.column_stack((x[::-1], bottom_curve[::-1]))
+        ])
+        
+        # Add slight padding for better coverage
+        center = np.mean(contour_points, axis=0)
+        vectors = contour_points - center
+        padded_points = center + vectors * 1.2  # Increased padding slightly
+        
+        return padded_points
+    return points
+
+def create_eyebrows_mask(face: Face, frame: Frame) -> (np.ndarray, np.ndarray, tuple, np.ndarray):
+    mask = np.zeros(frame.shape[:2], dtype=np.uint8)
+    eyebrows_cutout = None
+    landmarks = face.landmark_2d_106
+    if landmarks is not None:
+        # Left eyebrow landmarks (97-105) and right eyebrow landmarks (43-51)
+        left_eyebrow = landmarks[97:105].astype(np.float32)
+        right_eyebrow = landmarks[43:51].astype(np.float32)
+        
+        # Calculate centers and dimensions for each eyebrow
+        left_center = np.mean(left_eyebrow, axis=0)
+        right_center = np.mean(right_eyebrow, axis=0)
+        
+        # Calculate bounding box with padding adjusted by size
+        all_points = np.vstack([left_eyebrow, right_eyebrow])
+        padding_factor = modules.globals.eyebrows_mask_size
+        min_x = np.min(all_points[:, 0]) - 25 * padding_factor
+        max_x = np.max(all_points[:, 0]) + 25 * padding_factor
+        min_y = np.min(all_points[:, 1]) - 20 * padding_factor
+        max_y = np.max(all_points[:, 1]) + 15 * padding_factor
+        
+        # Ensure coordinates are within frame bounds
+        min_x = max(0, int(min_x))
+        min_y = max(0, int(min_y))
+        max_x = min(frame.shape[1], int(max_x))
+        max_y = min(frame.shape[0], int(max_y))
+        
+        # Create mask for the eyebrows region
+        mask_roi = np.zeros((max_y - min_y, max_x - min_x), dtype=np.uint8)
+        
+        try:
+            # Convert points to local coordinates
+            left_local = left_eyebrow - [min_x, min_y]
+            right_local = right_eyebrow - [min_x, min_y]
+            
+            def create_curved_eyebrow(points):
+                if len(points) >= 5:
+                    # Sort points by x-coordinate
+                    sorted_idx = np.argsort(points[:, 0])
+                    sorted_points = points[sorted_idx]
+                    
+                    # Calculate dimensions
+                    x_min, y_min = np.min(sorted_points, axis=0)
+                    x_max, y_max = np.max(sorted_points, axis=0)
+                    width = x_max - x_min
+                    height = y_max - y_min
+                    
+                    # Create more points for smoother curve
+                    num_points = 50
+                    x = np.linspace(x_min, x_max, num_points)
+                    
+                    # Fit quadratic curve through points for more natural arch
+                    coeffs = np.polyfit(sorted_points[:, 0], sorted_points[:, 1], 2)
+                    y = np.polyval(coeffs, x)
+                    
+                    # Increased offsets to create more separation
+                    top_offset = height * 0.5  # Increased from 0.3 to shift up more
+                    bottom_offset = height * 0.2  # Increased from 0.1 to shift down more
+                    
+                    # Create smooth curves
+                    top_curve = y - top_offset
+                    bottom_curve = y + bottom_offset
+                    
+                    # Create curved endpoints with more pronounced taper
+                    end_points = 5
+                    start_x = np.linspace(x[0] - width * 0.15, x[0], end_points)  # Increased taper
+                    end_x = np.linspace(x[-1], x[-1] + width * 0.15, end_points)  # Increased taper
+                    
+                    # Create tapered ends
+                    start_curve = np.column_stack((
+                        start_x,
+                        np.linspace(bottom_curve[0], top_curve[0], end_points)
+                    ))
+                    end_curve = np.column_stack((
+                        end_x,
+                        np.linspace(bottom_curve[-1], top_curve[-1], end_points)
+                    ))
+                    
+                    # Combine all points to form a smooth contour
+                    contour_points = np.vstack([
+                        start_curve,
+                        np.column_stack((x, top_curve)),
+                        end_curve,
+                        np.column_stack((x[::-1], bottom_curve[::-1]))
+                    ])
+                    
+                    # Add slight padding for better coverage
+                    center = np.mean(contour_points, axis=0)
+                    vectors = contour_points - center
+                    padded_points = center + vectors * 1.2  # Increased padding slightly
+                    
+                    return padded_points
+                return points
+            
+            # Generate and draw eyebrow shapes
+            left_shape = create_curved_eyebrow(left_local)
+            right_shape = create_curved_eyebrow(right_local)
+            
+            # Apply multi-stage blurring for natural feathering
+            # First, strong Gaussian blur for initial softening
+            mask_roi = cv2.GaussianBlur(mask_roi, (21, 21), 7)
+            
+            # Second, medium blur for transition areas
+            mask_roi = cv2.GaussianBlur(mask_roi, (11, 11), 3)
+            
+            # Finally, light blur for fine details
+            mask_roi = cv2.GaussianBlur(mask_roi, (5, 5), 1)
+            
+            # Normalize mask values
+            mask_roi = cv2.normalize(mask_roi, None, 0, 255, cv2.NORM_MINMAX)
+            
+            # Place the mask ROI in the full-sized mask
+            mask[min_y:max_y, min_x:max_x] = mask_roi
+            
+            # Extract the masked area from the frame
+            eyebrows_cutout = frame[min_y:max_y, min_x:max_x].copy()
+            
+            # Combine points for visualization
+            eyebrows_polygon = np.vstack([
+                left_shape + [min_x, min_y],
+                right_shape + [min_x, min_y]
+            ]).astype(np.int32)
+            
+        except Exception as e:
+            # Fallback to simple polygons if curve fitting fails
+            left_local = left_eyebrow - [min_x, min_y]
+            right_local = right_eyebrow - [min_x, min_y]
+            cv2.fillPoly(mask_roi, [left_local.astype(np.int32)], 255)
+            cv2.fillPoly(mask_roi, [right_local.astype(np.int32)], 255)
+            mask_roi = cv2.GaussianBlur(mask_roi, (21, 21), 7)
+            mask[min_y:max_y, min_x:max_x] = mask_roi
+            eyebrows_cutout = frame[min_y:max_y, min_x:max_x].copy()
+            eyebrows_polygon = np.vstack([left_eyebrow, right_eyebrow]).astype(np.int32)
+        
+    return mask, eyebrows_cutout, (min_x, min_y, max_x, max_y), eyebrows_polygon
+
+def apply_mask_area(
+    frame: np.ndarray,
+    cutout: np.ndarray,
+    box: tuple,
+    face_mask: np.ndarray,
+    polygon: np.ndarray,
+) -> np.ndarray:
+    min_x, min_y, max_x, max_y = box
+    box_width = max_x - min_x
+    box_height = max_y - min_y
+
+    if (
+        cutout is None
+        or box_width is None
+        or box_height is None
+        or face_mask is None
+        or polygon is None
+    ):
+        return frame
+
+    try:
+        resized_cutout = cv2.resize(cutout, (box_width, box_height))
+        roi = frame[min_y:max_y, min_x:max_x]
+
+        if roi.shape != resized_cutout.shape:
+            resized_cutout = cv2.resize(
+                resized_cutout, (roi.shape[1], roi.shape[0])
+            )
+
+        color_corrected_area = apply_color_transfer(resized_cutout, roi)
+
+        # Create mask for the area
+        polygon_mask = np.zeros(roi.shape[:2], dtype=np.uint8)
+        
+        # Split points for left and right parts if needed
+        if len(polygon) > 50:  # Arbitrary threshold to detect if we have multiple parts
+            mid_point = len(polygon) // 2
+            left_points = polygon[:mid_point] - [min_x, min_y]
+            right_points = polygon[mid_point:] - [min_x, min_y]
+            cv2.fillPoly(polygon_mask, [left_points], 255)
+            cv2.fillPoly(polygon_mask, [right_points], 255)
+        else:
+            adjusted_polygon = polygon - [min_x, min_y]
+            cv2.fillPoly(polygon_mask, [adjusted_polygon], 255)
+
+        # Apply strong initial feathering
+        polygon_mask = cv2.GaussianBlur(polygon_mask, (21, 21), 7)
+
+        # Apply additional feathering
+        feather_amount = min(
+            30,
+            box_width // modules.globals.mask_feather_ratio,
+            box_height // modules.globals.mask_feather_ratio,
+        )
+        feathered_mask = cv2.GaussianBlur(
+            polygon_mask.astype(float), (0, 0), feather_amount
+        )
+        feathered_mask = feathered_mask / feathered_mask.max()
+
+        # Apply additional smoothing to the mask edges
+        feathered_mask = cv2.GaussianBlur(feathered_mask, (5, 5), 1)
+
+        face_mask_roi = face_mask[min_y:max_y, min_x:max_x]
+        combined_mask = feathered_mask * (face_mask_roi / 255.0)
+
+        combined_mask = combined_mask[:, :, np.newaxis]
+        blended = (
+            color_corrected_area * combined_mask + roi * (1 - combined_mask)
+        ).astype(np.uint8)
+
+        # Apply face mask to blended result
+        face_mask_3channel = (
+            np.repeat(face_mask_roi[:, :, np.newaxis], 3, axis=2) / 255.0
+        )
+        final_blend = blended * face_mask_3channel + roi * (1 - face_mask_3channel)
+
+        frame[min_y:max_y, min_x:max_x] = final_blend.astype(np.uint8)
+    except Exception as e:
+        pass
+
+    return frame
+
+def draw_mask_visualization(
+    frame: Frame,
+    mask_data: tuple,
+    label: str,
+    draw_method: str = "polygon"
+) -> Frame:
+    mask, cutout, (min_x, min_y, max_x, max_y), polygon = mask_data
+
+    vis_frame = frame.copy()
+
+    # Ensure coordinates are within frame bounds
+    height, width = vis_frame.shape[:2]
+    min_x, min_y = max(0, min_x), max(0, min_y)
+    max_x, max_y = min(width, max_x), min(height, max_y)
+
+    if draw_method == "ellipse" and len(polygon) > 50:  # For eyes
+        # Split points for left and right parts
+        mid_point = len(polygon) // 2
+        left_points = polygon[:mid_point]
+        right_points = polygon[mid_point:]
+        
+        try:
+            # Fit ellipses to points - need at least 5 points
+            if len(left_points) >= 5 and len(right_points) >= 5:
+                # Convert points to the correct format for ellipse fitting
+                left_points = left_points.astype(np.float32)
+                right_points = right_points.astype(np.float32)
+                
+                # Fit ellipses
+                left_ellipse = cv2.fitEllipse(left_points)
+                right_ellipse = cv2.fitEllipse(right_points)
+                
+                # Draw the ellipses
+                cv2.ellipse(vis_frame, left_ellipse, (0, 255, 0), 2)
+                cv2.ellipse(vis_frame, right_ellipse, (0, 255, 0), 2)
+        except Exception as e:
+            # If ellipse fitting fails, draw simple rectangles as fallback
+            left_rect = cv2.boundingRect(left_points)
+            right_rect = cv2.boundingRect(right_points)
+            cv2.rectangle(vis_frame, 
+                        (left_rect[0], left_rect[1]), 
+                        (left_rect[0] + left_rect[2], left_rect[1] + left_rect[3]), 
+                        (0, 255, 0), 2)
+            cv2.rectangle(vis_frame,
+                        (right_rect[0], right_rect[1]),
+                        (right_rect[0] + right_rect[2], right_rect[1] + right_rect[3]),
+                        (0, 255, 0), 2)
+    else:  # For mouth and eyebrows
+        # Draw the polygon
+        if len(polygon) > 50:  # If we have multiple parts
+            mid_point = len(polygon) // 2
+            left_points = polygon[:mid_point]
+            right_points = polygon[mid_point:]
+            cv2.polylines(vis_frame, [left_points], True, (0, 255, 0), 2, cv2.LINE_AA)
+            cv2.polylines(vis_frame, [right_points], True, (0, 255, 0), 2, cv2.LINE_AA)
+        else:
+            cv2.polylines(vis_frame, [polygon], True, (0, 255, 0), 2, cv2.LINE_AA)
+
+    # Add label
+    cv2.putText(
+        vis_frame,
+        label,
+        (min_x, min_y - 10),
+        cv2.FONT_HERSHEY_SIMPLEX,
+        0.5,
+        (255, 255, 255),
+        1,
+    )
+
+    return vis_frame 

modules/processors/frame/face_swapper.py

@@ -14,6 +14,14 @@ from modules.utilities import (
     is_video,
 )
 from modules.cluster_analysis import find_closest_centroid
+from modules.processors.frame.face_masking import (
+    create_face_mask,
+    create_lower_mouth_mask,
+    create_eyes_mask,
+    create_eyebrows_mask,
+    apply_mask_area,
+    draw_mask_visualization
+)
 import os
 
 FACE_SWAPPER = None
@@ -74,24 +82,62 @@ def swap_face(source_face: Face, target_face: Face, temp_frame: Frame) -> Frame:
         temp_frame, target_face, source_face, paste_back=True
     )
 
-    if modules.globals.mouth_mask:
-        # Create a mask for the target face
+    # Create face mask for both mouth and eyes masking
     face_mask = create_face_mask(target_face, temp_frame)
 
-        # Create the mouth mask
-        mouth_mask, mouth_cutout, mouth_box, lower_lip_polygon = (
-            create_lower_mouth_mask(target_face, temp_frame)
-        )
-
-        # Apply the mouth area
-        swapped_frame = apply_mouth_area(
-            swapped_frame, mouth_cutout, mouth_box, face_mask, lower_lip_polygon
+    if modules.globals.mouth_mask:
+        # Create and apply mouth mask
+        mouth_mask_data = create_lower_mouth_mask(target_face, temp_frame)
+        swapped_frame = apply_mask_area(
+            swapped_frame, 
+            mouth_mask_data[1],  # mouth_cutout
+            mouth_mask_data[2],  # mouth_box
+            face_mask, 
+            mouth_mask_data[3]   # mouth_polygon
         )
 
         if modules.globals.show_mouth_mask_box:
-            mouth_mask_data = (mouth_mask, mouth_cutout, mouth_box, lower_lip_polygon)
-            swapped_frame = draw_mouth_mask_visualization(
-                swapped_frame, target_face, mouth_mask_data
+            swapped_frame = draw_mask_visualization(
+                swapped_frame, 
+                mouth_mask_data,
+                "Lower Mouth Mask"
+            )
+
+    if modules.globals.eyes_mask:
+        # Create and apply eyes mask
+        eyes_mask_data = create_eyes_mask(target_face, temp_frame)
+        swapped_frame = apply_mask_area(
+            swapped_frame, 
+            eyes_mask_data[1],  # eyes_cutout
+            eyes_mask_data[2],  # eyes_box
+            face_mask, 
+            eyes_mask_data[3]   # eyes_polygon
+        )
+
+        if modules.globals.show_eyes_mask_box:
+            swapped_frame = draw_mask_visualization(
+                swapped_frame, 
+                eyes_mask_data,
+                "Eyes Mask",
+                draw_method="ellipse"
+            )
+
+    if modules.globals.eyebrows_mask:
+        # Create and apply eyebrows mask
+        eyebrows_mask_data = create_eyebrows_mask(target_face, temp_frame)
+        swapped_frame = apply_mask_area(
+            swapped_frame, 
+            eyebrows_mask_data[1],  # eyebrows_cutout
+            eyebrows_mask_data[2],  # eyebrows_box
+            face_mask, 
+            eyebrows_mask_data[3]   # eyebrows_polygon
+        )
+
+        if modules.globals.show_eyebrows_mask_box:
+            swapped_frame = draw_mask_visualization(
+                swapped_frame, 
+                eyebrows_mask_data,
+                "Eyebrows Mask"
             )
 
     return swapped_frame
@@ -255,361 +301,3 @@ def process_video(source_path: str, temp_frame_paths: List[str]) -> None:
     modules.processors.frame.core.process_video(
         source_path, temp_frame_paths, process_frames
     )
-
-
-def create_lower_mouth_mask(
-    face: Face, frame: Frame
-) -> (np.ndarray, np.ndarray, tuple, np.ndarray):
-    mask = np.zeros(frame.shape[:2], dtype=np.uint8)
-    mouth_cutout = None
-    landmarks = face.landmark_2d_106
-    if landmarks is not None:
-        #                  0  1  2  3  4  5  6  7  8  9  10 11 12 13 14 15 16 17 18 19 20
-        lower_lip_order = [
-            65,
-            66,
-            62,
-            70,
-            69,
-            18,
-            19,
-            20,
-            21,
-            22,
-            23,
-            24,
-            0,
-            8,
-            7,
-            6,
-            5,
-            4,
-            3,
-            2,
-            65,
-        ]
-        lower_lip_landmarks = landmarks[lower_lip_order].astype(
-            np.float32
-        )  # Use float for precise calculations
-
-        # Calculate the center of the landmarks
-        center = np.mean(lower_lip_landmarks, axis=0)
-
-        # Expand the landmarks outward
-        expansion_factor = (
-            1 + modules.globals.mask_down_size
-        )  # Adjust this for more or less expansion
-        expanded_landmarks = (lower_lip_landmarks - center) * expansion_factor + center
-
-        # Extend the top lip part
-        toplip_indices = [
-            20,
-            0,
-            1,
-            2,
-            3,
-            4,
-            5,
-        ]  # Indices for landmarks 2, 65, 66, 62, 70, 69, 18
-        toplip_extension = (
-            modules.globals.mask_size * 0.5
-        )  # Adjust this factor to control the extension
-        for idx in toplip_indices:
-            direction = expanded_landmarks[idx] - center
-            direction = direction / np.linalg.norm(direction)
-            expanded_landmarks[idx] += direction * toplip_extension
-
-        # Extend the bottom part (chin area)
-        chin_indices = [
-            11,
-            12,
-            13,
-            14,
-            15,
-            16,
-        ]  # Indices for landmarks 21, 22, 23, 24, 0, 8
-        chin_extension = 2 * 0.2  # Adjust this factor to control the extension
-        for idx in chin_indices:
-            expanded_landmarks[idx][1] += (
-                expanded_landmarks[idx][1] - center[1]
-            ) * chin_extension
-
-        # Convert back to integer coordinates
-        expanded_landmarks = expanded_landmarks.astype(np.int32)
-
-        # Calculate bounding box for the expanded lower mouth
-        min_x, min_y = np.min(expanded_landmarks, axis=0)
-        max_x, max_y = np.max(expanded_landmarks, axis=0)
-
-        # Add some padding to the bounding box
-        padding = int((max_x - min_x) * 0.1)  # 10% padding
-        min_x = max(0, min_x - padding)
-        min_y = max(0, min_y - padding)
-        max_x = min(frame.shape[1], max_x + padding)
-        max_y = min(frame.shape[0], max_y + padding)
-
-        # Ensure the bounding box dimensions are valid
-        if max_x <= min_x or max_y <= min_y:
-            if (max_x - min_x) <= 1:
-                max_x = min_x + 1
-            if (max_y - min_y) <= 1:
-                max_y = min_y + 1
-
-        # Create the mask
-        mask_roi = np.zeros((max_y - min_y, max_x - min_x), dtype=np.uint8)
-        cv2.fillPoly(mask_roi, [expanded_landmarks - [min_x, min_y]], 255)
-
-        # Apply Gaussian blur to soften the mask edges
-        mask_roi = cv2.GaussianBlur(mask_roi, (15, 15), 5)
-
-        # Place the mask ROI in the full-sized mask
-        mask[min_y:max_y, min_x:max_x] = mask_roi
-
-        # Extract the masked area from the frame
-        mouth_cutout = frame[min_y:max_y, min_x:max_x].copy()
-
-        # Return the expanded lower lip polygon in original frame coordinates
-        lower_lip_polygon = expanded_landmarks
-
-    return mask, mouth_cutout, (min_x, min_y, max_x, max_y), lower_lip_polygon
-
-
-def draw_mouth_mask_visualization(
-    frame: Frame, face: Face, mouth_mask_data: tuple
-) -> Frame:
-    landmarks = face.landmark_2d_106
-    if landmarks is not None and mouth_mask_data is not None:
-        mask, mouth_cutout, (min_x, min_y, max_x, max_y), lower_lip_polygon = (
-            mouth_mask_data
-        )
-
-        vis_frame = frame.copy()
-
-        # Ensure coordinates are within frame bounds
-        height, width = vis_frame.shape[:2]
-        min_x, min_y = max(0, min_x), max(0, min_y)
-        max_x, max_y = min(width, max_x), min(height, max_y)
-
-        # Adjust mask to match the region size
-        mask_region = mask[0 : max_y - min_y, 0 : max_x - min_x]
-
-        # Remove the color mask overlay
-        # color_mask = cv2.applyColorMap((mask_region * 255).astype(np.uint8), cv2.COLORMAP_JET)
-
-        # Ensure shapes match before blending
-        vis_region = vis_frame[min_y:max_y, min_x:max_x]
-        # Remove blending with color_mask
-        # if vis_region.shape[:2] == color_mask.shape[:2]:
-        #     blended = cv2.addWeighted(vis_region, 0.7, color_mask, 0.3, 0)
-        #     vis_frame[min_y:max_y, min_x:max_x] = blended
-
-        # Draw the lower lip polygon
-        cv2.polylines(vis_frame, [lower_lip_polygon], True, (0, 255, 0), 2)
-
-        # Remove the red box
-        # cv2.rectangle(vis_frame, (min_x, min_y), (max_x, max_y), (0, 0, 255), 2)
-
-        # Visualize the feathered mask
-        feather_amount = max(
-            1,
-            min(
-                30,
-                (max_x - min_x) // modules.globals.mask_feather_ratio,
-                (max_y - min_y) // modules.globals.mask_feather_ratio,
-            ),
-        )
-        # Ensure kernel size is odd
-        kernel_size = 2 * feather_amount + 1
-        feathered_mask = cv2.GaussianBlur(
-            mask_region.astype(float), (kernel_size, kernel_size), 0
-        )
-        feathered_mask = (feathered_mask / feathered_mask.max() * 255).astype(np.uint8)
-        # Remove the feathered mask color overlay
-        # color_feathered_mask = cv2.applyColorMap(feathered_mask, cv2.COLORMAP_VIRIDIS)
-
-        # Ensure shapes match before blending feathered mask
-        # if vis_region.shape == color_feathered_mask.shape:
-        #     blended_feathered = cv2.addWeighted(vis_region, 0.7, color_feathered_mask, 0.3, 0)
-        #     vis_frame[min_y:max_y, min_x:max_x] = blended_feathered
-
-        # Add labels
-        cv2.putText(
-            vis_frame,
-            "Lower Mouth Mask",
-            (min_x, min_y - 10),
-            cv2.FONT_HERSHEY_SIMPLEX,
-            0.5,
-            (255, 255, 255),
-            1,
-        )
-        cv2.putText(
-            vis_frame,
-            "Feathered Mask",
-            (min_x, max_y + 20),
-            cv2.FONT_HERSHEY_SIMPLEX,
-            0.5,
-            (255, 255, 255),
-            1,
-        )
-
-        return vis_frame
-    return frame
-
-
-def apply_mouth_area(
-    frame: np.ndarray,
-    mouth_cutout: np.ndarray,
-    mouth_box: tuple,
-    face_mask: np.ndarray,
-    mouth_polygon: np.ndarray,
-) -> np.ndarray:
-    min_x, min_y, max_x, max_y = mouth_box
-    box_width = max_x - min_x
-    box_height = max_y - min_y
-
-    if (
-        mouth_cutout is None
-        or box_width is None
-        or box_height is None
-        or face_mask is None
-        or mouth_polygon is None
-    ):
-        return frame
-
-    try:
-        resized_mouth_cutout = cv2.resize(mouth_cutout, (box_width, box_height))
-        roi = frame[min_y:max_y, min_x:max_x]
-
-        if roi.shape != resized_mouth_cutout.shape:
-            resized_mouth_cutout = cv2.resize(
-                resized_mouth_cutout, (roi.shape[1], roi.shape[0])
-            )
-
-        color_corrected_mouth = apply_color_transfer(resized_mouth_cutout, roi)
-
-        # Use the provided mouth polygon to create the mask
-        polygon_mask = np.zeros(roi.shape[:2], dtype=np.uint8)
-        adjusted_polygon = mouth_polygon - [min_x, min_y]
-        cv2.fillPoly(polygon_mask, [adjusted_polygon], 255)
-
-        # Apply feathering to the polygon mask
-        feather_amount = min(
-            30,
-            box_width // modules.globals.mask_feather_ratio,
-            box_height // modules.globals.mask_feather_ratio,
-        )
-        feathered_mask = cv2.GaussianBlur(
-            polygon_mask.astype(float), (0, 0), feather_amount
-        )
-        feathered_mask = feathered_mask / feathered_mask.max()
-
-        face_mask_roi = face_mask[min_y:max_y, min_x:max_x]
-        combined_mask = feathered_mask * (face_mask_roi / 255.0)
-
-        combined_mask = combined_mask[:, :, np.newaxis]
-        blended = (
-            color_corrected_mouth * combined_mask + roi * (1 - combined_mask)
-        ).astype(np.uint8)
-
-        # Apply face mask to blended result
-        face_mask_3channel = (
-            np.repeat(face_mask_roi[:, :, np.newaxis], 3, axis=2) / 255.0
-        )
-        final_blend = blended * face_mask_3channel + roi * (1 - face_mask_3channel)
-
-        frame[min_y:max_y, min_x:max_x] = final_blend.astype(np.uint8)
-    except Exception as e:
-        pass
-
-    return frame
-
-
-def create_face_mask(face: Face, frame: Frame) -> np.ndarray:
-    mask = np.zeros(frame.shape[:2], dtype=np.uint8)
-    landmarks = face.landmark_2d_106
-    if landmarks is not None:
-        # Convert landmarks to int32
-        landmarks = landmarks.astype(np.int32)
-
-        # Extract facial features
-        right_side_face = landmarks[0:16]
-        left_side_face = landmarks[17:32]
-        right_eye = landmarks[33:42]
-        right_eye_brow = landmarks[43:51]
-        left_eye = landmarks[87:96]
-        left_eye_brow = landmarks[97:105]
-
-        # Calculate forehead extension
-        right_eyebrow_top = np.min(right_eye_brow[:, 1])
-        left_eyebrow_top = np.min(left_eye_brow[:, 1])
-        eyebrow_top = min(right_eyebrow_top, left_eyebrow_top)
-
-        face_top = np.min([right_side_face[0, 1], left_side_face[-1, 1]])
-        forehead_height = face_top - eyebrow_top
-        extended_forehead_height = int(forehead_height * 5.0)  # Extend by 50%
-
-        # Create forehead points
-        forehead_left = right_side_face[0].copy()
-        forehead_right = left_side_face[-1].copy()
-        forehead_left[1] -= extended_forehead_height
-        forehead_right[1] -= extended_forehead_height
-
-        # Combine all points to create the face outline
-        face_outline = np.vstack(
-            [
-                [forehead_left],
-                right_side_face,
-                left_side_face[
-                    ::-1
-                ],  # Reverse left side to create a continuous outline
-                [forehead_right],
-            ]
-        )
-
-        # Calculate padding
-        padding = int(
-            np.linalg.norm(right_side_face[0] - left_side_face[-1]) * 0.05
-        )  # 5% of face width
-
-        # Create a slightly larger convex hull for padding
-        hull = cv2.convexHull(face_outline)
-        hull_padded = []
-        for point in hull:
-            x, y = point[0]
-            center = np.mean(face_outline, axis=0)
-            direction = np.array([x, y]) - center
-            direction = direction / np.linalg.norm(direction)
-            padded_point = np.array([x, y]) + direction * padding
-            hull_padded.append(padded_point)
-
-        hull_padded = np.array(hull_padded, dtype=np.int32)
-
-        # Fill the padded convex hull
-        cv2.fillConvexPoly(mask, hull_padded, 255)
-
-        # Smooth the mask edges
-        mask = cv2.GaussianBlur(mask, (5, 5), 3)
-
-    return mask
-
-
-def apply_color_transfer(source, target):
-    """
-    Apply color transfer from target to source image
-    """
-    source = cv2.cvtColor(source, cv2.COLOR_BGR2LAB).astype("float32")
-    target = cv2.cvtColor(target, cv2.COLOR_BGR2LAB).astype("float32")
-
-    source_mean, source_std = cv2.meanStdDev(source)
-    target_mean, target_std = cv2.meanStdDev(target)
-
-    # Reshape mean and std to be broadcastable
-    source_mean = source_mean.reshape(1, 1, 3)
-    source_std = source_std.reshape(1, 1, 3)
-    target_mean = target_mean.reshape(1, 1, 3)
-    target_std = target_std.reshape(1, 1, 3)
-
-    # Perform the color transfer
-    source = (source - source_mean) * (target_std / source_std) + target_mean
-
-    return cv2.cvtColor(np.clip(source, 0, 255).astype("uint8"), cv2.COLOR_LAB2BGR)

requirements.txt

@@ -8,7 +8,7 @@ insightface==0.7.3
 psutil==5.9.8
 tk==0.1.0
 customtkinter==5.2.2
-pillow==9.5.0
+pillow==11.1.0
 torch==2.0.1+cu118; sys_platform != 'darwin'
 torch==2.0.1; sys_platform == 'darwin'
 torchvision==0.15.2+cu118; sys_platform != 'darwin'