Compare commits

..

3 Commits

Author SHA1 Message Date
Makaru 5dd6d1fe64 Fixed 0 Transparency
The error was caused by an erroneous designation of "face_swapper_enabled" in lieu of "fp_ui."
2025-01-15 02:53:11 +08:00
Makaru 9af216e819 Opacity Update
- Added 0 value, if it is set to 0, the face swapping will be disabled
2025-01-14 22:45:16 +08:00
Makaru 59d64d4b6a Added dropdown transparency 2025-01-13 01:23:58 +08:00
9 changed files with 629 additions and 330 deletions
+33 -128
View File
@@ -9,85 +9,52 @@
</p>
<p align="center">
<img src="media/demo.gif" alt="Demo GIF" width="800">
<img src="media/demo.gif" alt="Demo GIF">
<img src="media/avgpcperformancedemo.gif" alt="Performance Demo GIF">
</p>
## Disclaimer
###### This software is intended as a productive contribution to the AI-generated media industry. It aims to assist artists with tasks like animating custom characters or using them as models for clothing, etc.
This deepfake software is designed to be a productive tool for the AI-generated media industry. It can assist artists in animating custom characters, creating engaging content, and even using models for clothing design.
###### We are aware of the potential for unethical applications and are committed to preventative measures. A built-in check prevents the program from processing inappropriate media (nudity, graphic content, sensitive material like war footage, etc.). We will continue to develop this project responsibly, adhering to the law and ethics. We may shut down the project or add watermarks if legally required.
We are aware of the potential for unethical applications and are committed to preventative measures. A built-in check prevents the program from processing inappropriate media (nudity, graphic content, sensitive material like war footage, etc.). We will continue to develop this project responsibly, adhering to the law and ethics. We may shut down the project or add watermarks if legally required.
###### Users are expected to use this software responsibly and legally. If using a real person's face, obtain their consent and clearly label any output as a deepfake when sharing online. We are not responsible for end-user actions.
- Ethical Use: Users are expected to use this software responsibly and legally. If using a real person's face, obtain their consent and clearly label any output as a deepfake when sharing online.
## Quick Start - Pre-built
<div align="center">
<a href="https://hacksider.gumroad.com/l/vccdmm">
<img src="https://github.com/user-attachments/assets/7d993b32-e3e8-4cd3-bbfb-a549152ebdd5" width="285" height="77" />
</a>
<a href="https://krshh.gumroad.com/l/Deep-Live-Cam-Mac">
<img src="https://github.com/user-attachments/assets/d5d913b5-a7de-4609-96b9-979a5749a703" width="285" height="77" />
</a>
</div>
- Content Restrictions: The software includes built-in checks to prevent processing inappropriate media, such as nudity, graphic content, or sensitive material.
- Legal Compliance: We adhere to all relevant laws and ethical guidelines. If legally required, we may shut down the project or add watermarks to the output.
- User Responsibility: We are not responsible for end-user actions. Users must ensure their use of the software aligns with ethical standards and legal requirements.
By using this software, you agree to these terms and commit to using it in a manner that respects the rights and dignity of others.
Users are expected to use this software responsibly and legally. If using a real person's face, obtain their consent and clearly label any output as a deepfake when sharing online. We are not responsible for end-user actions.
## TLDR; Live Deepfake in just 3 Clicks
![easysteps](https://github.com/user-attachments/assets/af825228-852c-411b-b787-ffd9aac72fc6)
1. Select a face
2. Select which camera to use
3. Press live!
## Features & Uses - Everything is in real-time
## Features - Everything is real-time
### Mouth Mask
**Retain your original mouth for accurate movement using Mouth Mask**
**Retain your original mouth using Mouth Mask**
<p align="center">
<img src="media/ludwig.gif" alt="resizable-gif">
</p>
![resizable-gif](media/ludwig.gif)
### Face Mapping
**Use different faces on multiple subjects simultaneously**
**Use different faces on multiple subjects**
<p align="center">
<img src="media/streamers.gif" alt="face_mapping_source">
</p>
![face\_mapping\_source](media/streamers.gif)
### Your Movie, Your Face
**Watch movies with any face in real-time**
<p align="center">
<img src="media/movie.gif" alt="movie">
</p>
![movie](media/movie.gif)
### Live Show
## Benchmarks
**Run Live shows and performances**
**Nearly 0% detection!**
<p align="center">
<img src="media/live_show.gif" alt="show">
</p>
### Memes
**Create Your Most Viral Meme Yet**
<p align="center">
<img src="media/meme.gif" alt="show" width="450">
<br>
<sub>Created using Many Faces feature in Deep-Live-Cam</sub>
</p>
### Omegle
**Surprise people on Omegle**
<p align="center">
<video src="https://github.com/user-attachments/assets/2e9b9b82-fa04-4b70-9f56-b1f68e7672d0" width="450" controls></video>
</p>
![bench](media/deepwarebench.gif)
## Installation (Manual)
@@ -105,20 +72,19 @@ This is more likely to work on your computer but will be slower as it utilizes t
- Python (3.10 recommended)
- pip
- git
- [ffmpeg](https://www.youtube.com/watch?v=OlNWCpFdVMA) - ```iex (irm ffmpeg.tc.ht)```
- [ffmpeg](https://www.youtube.com/watch?v=OlNWCpFdVMA)
- [Visual Studio 2022 Runtimes (Windows)](https://visualstudio.microsoft.com/visual-cpp-build-tools/)
**2. Clone the Repository**
```bash
git clone https://github.com/hacksider/Deep-Live-Cam.git
cd Deep-Live-Cam
https://github.com/hacksider/Deep-Live-Cam.git
```
**3. Download the Models**
1. [GFPGANv1.4](https://huggingface.co/hacksider/deep-live-cam/resolve/main/GFPGANv1.4.pth)
2. [inswapper\_128\_fp16.onnx](https://huggingface.co/hacksider/deep-live-cam/resolve/main/inswapper_128_fp16.onnx)
2. [inswapper\_128\_fp16.onnx](https://huggingface.co/hacksider/deep-live-cam/resolve/main/inswapper_128.onnx) (Note: Use this [replacement version](https://github.com/facefusion/facefusion-assets/releases/download/models/inswapper_128.onnx) if you encounter issues)
Place these files in the "**models**" folder.
@@ -126,44 +92,14 @@ Place these files in the "**models**" folder.
We highly recommend using a `venv` to avoid issues.
For Windows:
```bash
python -m venv venv
venv\Scripts\activate
pip install -r requirements.txt
```
**For macOS:**
Apple Silicon (M1/M2/M3) requires specific setup:
**For macOS:** Install or upgrade the `python-tk` package:
```bash
# Install Python 3.10 (specific version is important)
brew install python@3.10
# Install tkinter package (required for the GUI)
brew install python-tk@3.10
# Create and activate virtual environment with Python 3.10
python3.10 -m venv venv
source venv/bin/activate
# Install dependencies
pip install -r requirements.txt
```
** In case something goes wrong and you need to reinstall the virtual environment **
```bash
# Deactivate the virtual environment
rm -rf venv
# Reinstall the virtual environment
python -m venv venv
source venv/bin/activate
# install the dependencies again
pip install -r requirements.txt
```
**Run:** If you don't have a GPU, you can run Deep-Live-Cam using `python run.py`. Note that initial execution will download models (~300MB).
@@ -172,7 +108,7 @@ pip install -r requirements.txt
**CUDA Execution Provider (Nvidia)**
1. Install [CUDA Toolkit 11.8.0](https://developer.nvidia.com/cuda-11-8-0-download-archive)
1. Install [CUDA Toolkit 11.8](https://developer.nvidia.com/cuda-11-8-0-download-archive) or [CUDA Toolkit 12.1.1](https://developer.nvidia.com/cuda-12-1-1-download-archive)
2. Install dependencies:
```bash
@@ -188,39 +124,19 @@ python run.py --execution-provider cuda
**CoreML Execution Provider (Apple Silicon)**
Apple Silicon (M1/M2/M3) specific installation:
1. Make sure you've completed the macOS setup above using Python 3.10.
2. Install dependencies:
1. Install dependencies:
```bash
pip uninstall onnxruntime onnxruntime-silicon
pip install onnxruntime-silicon==1.13.1
```
3. Usage (important: specify Python 3.10):
2. Usage:
```bash
python3.10 run.py --execution-provider coreml
python run.py --execution-provider coreml
```
**Important Notes for macOS:**
- You **must** use Python 3.10, not newer versions like 3.11 or 3.13
- Always run with `python3.10` command not just `python` if you have multiple Python versions installed
- If you get error about `_tkinter` missing, reinstall the tkinter package: `brew reinstall python-tk@3.10`
- If you get model loading errors, check that your models are in the correct folder
- If you encounter conflicts with other Python versions, consider uninstalling them:
```bash
# List all installed Python versions
brew list | grep python
# Uninstall conflicting versions if needed
brew uninstall --ignore-dependencies python@3.11 python@3.13
# Keep only Python 3.10
brew cleanup
```
**CoreML Execution Provider (Apple Legacy)**
1. Install dependencies:
@@ -265,6 +181,7 @@ pip install onnxruntime-openvino==1.15.0
```bash
python run.py --execution-provider openvino
```
</details>
## Usage
@@ -285,19 +202,6 @@ python run.py --execution-provider openvino
- Use a screen capture tool like OBS to stream.
- To change the face, select a new source image.
## Tips and Tricks
Check out these helpful guides to get the most out of Deep-Live-Cam:
- [Unlocking the Secrets to the Perfect Deepfake Image](https://deeplivecam.net/index.php/blog/tips-and-tricks/unlocking-the-secrets-to-the-perfect-deepfake-image) - Learn how to create the best deepfake with full head coverage
- [Video Call with DeepLiveCam](https://deeplivecam.net/index.php/blog/tips-and-tricks/video-call-with-deeplivecam) - Make your meetings livelier by using DeepLiveCam with OBS and meeting software
- [Have a Special Guest!](https://deeplivecam.net/index.php/blog/tips-and-tricks/have-a-special-guest) - Tutorial on how to use face mapping to add special guests to your stream
- [Watch Deepfake Movies in Realtime](https://deeplivecam.net/index.php/blog/tips-and-tricks/watch-deepfake-movies-in-realtime) - See yourself star in any video without processing the video
- [Better Quality without Sacrificing Speed](https://deeplivecam.net/index.php/blog/tips-and-tricks/better-quality-without-sacrificing-speed) - Tips for achieving better results without impacting performance
- [Instant Vtuber!](https://deeplivecam.net/index.php/blog/tips-and-tricks/instant-vtuber) - Create a new persona/vtuber easily using Metahuman Creator
Visit our [official blog](https://deeplivecam.net/index.php/blog/tips-and-tricks) for more tips and tutorials.
## Command Line Arguments (Unmaintained)
```
@@ -313,6 +217,7 @@ options:
--many-faces process every face
--map-faces map source target faces
--mouth-mask mask the mouth region
--nsfw-filter filter the NSFW image or video
--video-encoder {libx264,libx265,libvpx-vp9} adjust output video encoder
--video-quality [0-51] adjust output video quality
--live-mirror the live camera display as you see it in the front-facing camera frame
Binary file not shown.

Before

Width:  |  Height:  |  Size: 8.2 MiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 5.0 MiB

+12 -12
View File
@@ -39,13 +39,13 @@ def get_many_faces(frame: Frame) -> Any:
return None
def has_valid_map() -> bool:
for map in modules.globals.source_target_map:
for map in modules.globals.souce_target_map:
if "source" in map and "target" in map:
return True
return False
def default_source_face() -> Any:
for map in modules.globals.source_target_map:
for map in modules.globals.souce_target_map:
if "source" in map:
return map['source']['face']
return None
@@ -53,7 +53,7 @@ def default_source_face() -> Any:
def simplify_maps() -> Any:
centroids = []
faces = []
for map in modules.globals.source_target_map:
for map in modules.globals.souce_target_map:
if "source" in map and "target" in map:
centroids.append(map['target']['face'].normed_embedding)
faces.append(map['source']['face'])
@@ -64,10 +64,10 @@ def simplify_maps() -> Any:
def add_blank_map() -> Any:
try:
max_id = -1
if len(modules.globals.source_target_map) > 0:
max_id = max(modules.globals.source_target_map, key=lambda x: x['id'])['id']
if len(modules.globals.souce_target_map) > 0:
max_id = max(modules.globals.souce_target_map, key=lambda x: x['id'])['id']
modules.globals.source_target_map.append({
modules.globals.souce_target_map.append({
'id' : max_id + 1
})
except ValueError:
@@ -75,14 +75,14 @@ def add_blank_map() -> Any:
def get_unique_faces_from_target_image() -> Any:
try:
modules.globals.source_target_map = []
modules.globals.souce_target_map = []
target_frame = cv2.imread(modules.globals.target_path)
many_faces = get_many_faces(target_frame)
i = 0
for face in many_faces:
x_min, y_min, x_max, y_max = face['bbox']
modules.globals.source_target_map.append({
modules.globals.souce_target_map.append({
'id' : i,
'target' : {
'cv2' : target_frame[int(y_min):int(y_max), int(x_min):int(x_max)],
@@ -96,7 +96,7 @@ def get_unique_faces_from_target_image() -> Any:
def get_unique_faces_from_target_video() -> Any:
try:
modules.globals.source_target_map = []
modules.globals.souce_target_map = []
frame_face_embeddings = []
face_embeddings = []
@@ -127,7 +127,7 @@ def get_unique_faces_from_target_video() -> Any:
face['target_centroid'] = closest_centroid_index
for i in range(len(centroids)):
modules.globals.source_target_map.append({
modules.globals.souce_target_map.append({
'id' : i
})
@@ -135,7 +135,7 @@ def get_unique_faces_from_target_video() -> Any:
for frame in tqdm(frame_face_embeddings, desc=f"Mapping frame embeddings to centroids-{i}"):
temp.append({'frame': frame['frame'], 'faces': [face for face in frame['faces'] if face['target_centroid'] == i], 'location': frame['location']})
modules.globals.source_target_map[i]['target_faces_in_frame'] = temp
modules.globals.souce_target_map[i]['target_faces_in_frame'] = temp
# dump_faces(centroids, frame_face_embeddings)
default_target_face()
@@ -144,7 +144,7 @@ def get_unique_faces_from_target_video() -> Any:
def default_target_face():
for map in modules.globals.source_target_map:
for map in modules.globals.souce_target_map:
best_face = None
best_frame = None
for frame in map['target_faces_in_frame']:
+3 -1
View File
@@ -9,7 +9,7 @@ file_types = [
("Video", ("*.mp4", "*.mkv")),
]
source_target_map = []
souce_target_map = []
simple_map = {}
source_path = None
@@ -41,3 +41,5 @@ show_mouth_mask_box = False
mask_feather_ratio = 8
mask_down_size = 0.50
mask_size = 1
opacity = 1.0
face_swapper_enabled = True
+1 -1
View File
@@ -1,3 +1,3 @@
name = 'Deep-Live-Cam'
version = '1.9'
version = '1.8'
edition = 'GitHub Edition'
+523 -157
View File
@@ -1,44 +1,55 @@
import os # <-- Added for os.path.exists
from typing import Any, List
import cv2
import insightface
import threading
import numpy as np
import modules.globals
import modules.processors.frame.core
# Ensure update_status is imported if not already globally accessible
# If it's part of modules.core, it might already be accessible via modules.core.update_status
from modules.core import update_status
from modules.face_analyser import get_one_face, get_many_faces, default_source_face
from modules.typing import Face, Frame
from modules.utilities import conditional_download, resolve_relative_path, is_image, is_video
from modules.utilities import (
conditional_download,
is_image,
is_video,
)
from modules.cluster_analysis import find_closest_centroid
from modules.globals import face_swapper_enabled, opacity
import os
FACE_SWAPPER = None
THREAD_LOCK = threading.Lock()
NAME = 'DLC.FACE-SWAPPER'
NAME = "DLC.FACE-SWAPPER"
abs_dir = os.path.dirname(os.path.abspath(__file__))
models_dir = os.path.join(
os.path.dirname(os.path.dirname(os.path.dirname(abs_dir))), "models"
)
def pre_check() -> bool:
download_directory_path = resolve_relative_path('../models')
# Ensure both models are mentioned or downloaded if necessary
# Conditional download might need adjustment if you want it to fetch FP32 too
conditional_download(download_directory_path, ['https://huggingface.co/hacksider/deep-live-cam/blob/main/inswapper_128_fp16.onnx'])
# Add a check or download for the FP32 model if you have a URL
# conditional_download(download_directory_path, ['URL_TO_FP32_MODEL_HERE'])
download_directory_path = abs_dir
conditional_download(
download_directory_path,
[
"https://huggingface.co/hacksider/deep-live-cam/blob/main/inswapper_128_fp16.onnx"
],
)
return True
def pre_start() -> bool:
# --- No changes needed in pre_start ---
if not modules.globals.map_faces and not is_image(modules.globals.source_path):
update_status('Select an image for source path.', NAME)
update_status("Select an image for source path.", NAME)
return False
elif not modules.globals.map_faces and not get_one_face(cv2.imread(modules.globals.source_path)):
update_status('No face in source path detected.', NAME)
elif not modules.globals.map_faces and not get_one_face(
cv2.imread(modules.globals.source_path)
):
update_status("No face in source path detected.", NAME)
return False
if not is_image(modules.globals.target_path) and not is_video(modules.globals.target_path):
update_status('Select an image or video for target path.', NAME)
if not is_image(modules.globals.target_path) and not is_video(
modules.globals.target_path
):
update_status("Select an image or video for target path.", NAME)
return False
return True
@@ -48,57 +59,53 @@ def get_face_swapper() -> Any:
with THREAD_LOCK:
if FACE_SWAPPER is None:
# --- MODIFICATION START ---
# Define paths for both FP32 and FP16 models
model_dir = resolve_relative_path('../models')
model_path_fp32 = os.path.join(model_dir, 'inswapper_128.onnx')
model_path_fp16 = os.path.join(model_dir, 'inswapper_128_fp16.onnx')
chosen_model_path = None
# Prioritize FP32 model
if os.path.exists(model_path_fp32):
chosen_model_path = model_path_fp32
update_status(f"Loading FP32 model: {os.path.basename(chosen_model_path)}", NAME)
# Fallback to FP16 model
elif os.path.exists(model_path_fp16):
chosen_model_path = model_path_fp16
update_status(f"FP32 model not found. Loading FP16 model: {os.path.basename(chosen_model_path)}", NAME)
# Error if neither model is found
else:
error_message = f"Face Swapper model not found. Please ensure 'inswapper_128.onnx' (recommended) or 'inswapper_128_fp16.onnx' exists in the '{model_dir}' directory."
update_status(error_message, NAME)
raise FileNotFoundError(error_message)
# Load the chosen model
try:
FACE_SWAPPER = insightface.model_zoo.get_model(chosen_model_path, providers=modules.globals.execution_providers)
except Exception as e:
update_status(f"Error loading Face Swapper model {os.path.basename(chosen_model_path)}: {e}", NAME)
# Optionally, re-raise the exception or handle it more gracefully
raise e
# --- MODIFICATION END ---
model_path = os.path.join(models_dir, "inswapper_128_fp16.onnx")
FACE_SWAPPER = insightface.model_zoo.get_model(
model_path, providers=modules.globals.execution_providers
)
return FACE_SWAPPER
def swap_face(source_face: Face, target_face: Face, temp_frame: Frame) -> Frame:
# --- No changes needed in swap_face ---
swapper = get_face_swapper()
if swapper is None:
# Handle case where model failed to load
update_status("Face swapper model not loaded, skipping swap.", NAME)
return temp_frame
return swapper.get(temp_frame, target_face, source_face, paste_back=True)
face_swapper = get_face_swapper()
# Apply the face swap
swapped_frame = face_swapper.get(
temp_frame, target_face, source_face, paste_back=True
)
if modules.globals.mouth_mask:
# Create a mask for the target face
face_mask = create_face_mask(target_face, temp_frame)
# Create the mouth mask
mouth_mask, mouth_cutout, mouth_box, lower_lip_polygon = (
create_lower_mouth_mask(target_face, temp_frame)
)
# Apply the mouth area
swapped_frame = apply_mouth_area(
swapped_frame, mouth_cutout, mouth_box, face_mask, lower_lip_polygon
)
if modules.globals.show_mouth_mask_box:
mouth_mask_data = (mouth_mask, mouth_cutout, mouth_box, lower_lip_polygon)
swapped_frame = draw_mouth_mask_visualization(
swapped_frame, target_face, mouth_mask_data
)
opacity = getattr(modules.globals, "opacity", 1.0)
swapped_frame = cv2.addWeighted(temp_frame, 1 - opacity, swapped_frame, opacity, 0)
return swapped_frame
def process_frame(source_face: Face, temp_frame: Frame) -> Frame:
# --- No changes needed in process_frame ---
# Ensure the frame is in RGB format if color correction is enabled
# Note: InsightFace swapper often expects BGR by default. Double-check if color issues appear.
# If color correction is needed *before* swapping and insightface needs BGR:
# original_was_bgr = True # Assume input is BGR
# if modules.globals.color_correction:
# temp_frame = cv2.cvtColor(temp_frame, cv2.COLOR_BGR2RGB)
# original_was_bgr = False # Now it's RGB
if getattr(modules.globals, "opacity", 1.0) == 0:
return temp_frame
if modules.globals.color_correction:
temp_frame = cv2.cvtColor(temp_frame, cv2.COLOR_BGR2RGB)
if modules.globals.many_faces:
many_faces = get_many_faces(temp_frame)
@@ -109,51 +116,56 @@ def process_frame(source_face: Face, temp_frame: Frame) -> Frame:
target_face = get_one_face(temp_frame)
if target_face:
temp_frame = swap_face(source_face, target_face, temp_frame)
# Convert back if necessary (example, might not be needed depending on workflow)
# if modules.globals.color_correction and not original_was_bgr:
# temp_frame = cv2.cvtColor(temp_frame, cv2.COLOR_RGB2BGR)
return temp_frame
def process_frame_v2(temp_frame: Frame, temp_frame_path: str = "") -> Frame:
# --- No changes needed in process_frame_v2 ---
# (Assuming swap_face handles the potential None return from get_face_swapper)
if getattr(modules.globals, "opacity", 1.0) == 0:
return temp_frame
if is_image(modules.globals.target_path):
if modules.globals.many_faces:
source_face = default_source_face()
for map_entry in modules.globals.souce_target_map: # Renamed 'map' to 'map_entry'
target_face = map_entry['target']['face']
for map in modules.globals.souce_target_map:
target_face = map["target"]["face"]
temp_frame = swap_face(source_face, target_face, temp_frame)
elif not modules.globals.many_faces:
for map_entry in modules.globals.souce_target_map: # Renamed 'map' to 'map_entry'
if "source" in map_entry:
source_face = map_entry['source']['face']
target_face = map_entry['target']['face']
for map in modules.globals.souce_target_map:
if "source" in map:
source_face = map["source"]["face"]
target_face = map["target"]["face"]
temp_frame = swap_face(source_face, target_face, temp_frame)
elif is_video(modules.globals.target_path):
if modules.globals.many_faces:
source_face = default_source_face()
for map_entry in modules.globals.souce_target_map: # Renamed 'map' to 'map_entry'
target_frame = [f for f in map_entry['target_faces_in_frame'] if f['location'] == temp_frame_path]
for map in modules.globals.souce_target_map:
target_frame = [
f
for f in map["target_faces_in_frame"]
if f["location"] == temp_frame_path
]
for frame in target_frame:
for target_face in frame['faces']:
for target_face in frame["faces"]:
temp_frame = swap_face(source_face, target_face, temp_frame)
elif not modules.globals.many_faces:
for map_entry in modules.globals.souce_target_map: # Renamed 'map' to 'map_entry'
if "source" in map_entry:
target_frame = [f for f in map_entry['target_faces_in_frame'] if f['location'] == temp_frame_path]
source_face = map_entry['source']['face']
for map in modules.globals.souce_target_map:
if "source" in map:
target_frame = [
f
for f in map["target_faces_in_frame"]
if f["location"] == temp_frame_path
]
source_face = map["source"]["face"]
for frame in target_frame:
for target_face in frame['faces']:
for target_face in frame["faces"]:
temp_frame = swap_face(source_face, target_face, temp_frame)
else: # Fallback for neither image nor video (e.g., live feed?)
else:
detected_faces = get_many_faces(temp_frame)
if modules.globals.many_faces:
if detected_faces:
@@ -162,97 +174,451 @@ def process_frame_v2(temp_frame: Frame, temp_frame_path: str = "") -> Frame:
temp_frame = swap_face(source_face, target_face, temp_frame)
elif not modules.globals.many_faces:
if detected_faces and hasattr(modules.globals, 'simple_map') and modules.globals.simple_map: # Check simple_map exists
if len(detected_faces) <= len(modules.globals.simple_map['target_embeddings']):
if detected_faces:
if len(detected_faces) <= len(
modules.globals.simple_map["target_embeddings"]
):
for detected_face in detected_faces:
closest_centroid_index, _ = find_closest_centroid(modules.globals.simple_map['target_embeddings'], detected_face.normed_embedding)
temp_frame = swap_face(modules.globals.simple_map['source_faces'][closest_centroid_index], detected_face, temp_frame)
closest_centroid_index, _ = find_closest_centroid(
modules.globals.simple_map["target_embeddings"],
detected_face.normed_embedding,
)
temp_frame = swap_face(
modules.globals.simple_map["source_faces"][
closest_centroid_index
],
detected_face,
temp_frame,
)
else:
detected_faces_centroids = [face.normed_embedding for face in detected_faces]
detected_faces_centroids = []
for face in detected_faces:
detected_faces_centroids.append(face.normed_embedding)
i = 0
for target_embedding in modules.globals.simple_map['target_embeddings']:
closest_centroid_index, _ = find_closest_centroid(detected_faces_centroids, target_embedding)
# Ensure index is valid before accessing detected_faces
if closest_centroid_index < len(detected_faces):
temp_frame = swap_face(modules.globals.simple_map['source_faces'][i], detected_faces[closest_centroid_index], temp_frame)
for target_embedding in modules.globals.simple_map[
"target_embeddings"
]:
closest_centroid_index, _ = find_closest_centroid(
detected_faces_centroids, target_embedding
)
temp_frame = swap_face(
modules.globals.simple_map["source_faces"][i],
detected_faces[closest_centroid_index],
temp_frame,
)
i += 1
return temp_frame
def process_frames(source_path: str, temp_frame_paths: List[str], progress: Any = None) -> None:
# --- No changes needed in process_frames ---
# Note: Ensure get_one_face is called only once if possible for efficiency if !map_faces
source_face = None
def process_frames(
source_path: str, temp_frame_paths: List[str], progress: Any = None
) -> None:
if not modules.globals.map_faces:
source_img = cv2.imread(source_path)
if source_img is not None:
source_face = get_one_face(source_img)
if source_face is None:
update_status(f"Could not find face in source image: {source_path}, skipping swap.", NAME)
# If no source face, maybe skip processing? Or handle differently.
# For now, it will proceed but swap_face might fail later.
for temp_frame_path in temp_frame_paths:
temp_frame = cv2.imread(temp_frame_path)
if temp_frame is None:
update_status(f"Warning: Could not read frame {temp_frame_path}", NAME)
if progress: progress.update(1) # Still update progress even if frame fails
continue # Skip to next frame
try:
if not modules.globals.map_faces:
if source_face: # Only process if source face was found
result = process_frame(source_face, temp_frame)
else:
result = temp_frame # No source face, return original frame
else:
result = process_frame_v2(temp_frame, temp_frame_path)
cv2.imwrite(temp_frame_path, result)
except Exception as exception:
update_status(f"Error processing frame {os.path.basename(temp_frame_path)}: {exception}", NAME)
# Decide whether to 'pass' (continue processing other frames) or raise
pass # Continue processing other frames
finally:
source_face = get_one_face(cv2.imread(source_path))
for temp_frame_path in temp_frame_paths:
temp_frame = cv2.imread(temp_frame_path)
try:
result = process_frame(source_face, temp_frame)
cv2.imwrite(temp_frame_path, result)
except Exception as exception:
print(exception)
pass
if progress:
progress.update(1)
else:
for temp_frame_path in temp_frame_paths:
temp_frame = cv2.imread(temp_frame_path)
try:
result = process_frame_v2(temp_frame, temp_frame_path)
cv2.imwrite(temp_frame_path, result)
except Exception as exception:
print(exception)
pass
if progress:
progress.update(1)
def process_image(source_path: str, target_path: str, output_path: str) -> None:
# --- No changes needed in process_image ---
# Note: Added checks for successful image reads and face detection
target_frame = cv2.imread(target_path) # Read original target for processing
if target_frame is None:
update_status(f"Error: Could not read target image: {target_path}", NAME)
return
if not modules.globals.map_faces:
source_img = cv2.imread(source_path)
if source_img is None:
update_status(f"Error: Could not read source image: {source_path}", NAME)
return
source_face = get_one_face(source_img)
if source_face is None:
update_status(f"Error: No face found in source image: {source_path}", NAME)
return
source_face = get_one_face(cv2.imread(source_path))
target_frame = cv2.imread(target_path)
result = process_frame(source_face, target_frame)
cv2.imwrite(output_path, result)
else:
if modules.globals.many_faces:
update_status('Many faces enabled. Using first source image (if applicable in v2). Processing...', NAME)
# For process_frame_v2 on single image, it reads the 'output_path' which should be a copy
# Let's process the 'target_frame' we read instead.
result = process_frame_v2(target_frame) # Process the frame directly
# Write the final result to the output path
success = cv2.imwrite(output_path, result)
if not success:
update_status(f"Error: Failed to write output image to: {output_path}", NAME)
update_status(
"Many faces enabled. Using first source image. Progressing...", NAME
)
target_frame = cv2.imread(output_path)
result = process_frame_v2(target_frame)
cv2.imwrite(output_path, result)
def process_video(source_path: str, temp_frame_paths: List[str]) -> None:
# --- No changes needed in process_video ---
if modules.globals.map_faces and modules.globals.many_faces:
update_status('Many faces enabled. Using first source image (if applicable in v2). Processing...', NAME)
# The core processing logic is delegated, which is good.
modules.processors.frame.core.process_video(source_path, temp_frame_paths, process_frames)
update_status(
"Many faces enabled. Using first source image. Progressing...", NAME
)
modules.processors.frame.core.process_video(
source_path, temp_frame_paths, process_frames
)
def create_lower_mouth_mask(
face: Face, frame: Frame
) -> (np.ndarray, np.ndarray, tuple, np.ndarray):
mask = np.zeros(frame.shape[:2], dtype=np.uint8)
mouth_cutout = None
landmarks = face.landmark_2d_106
if landmarks is not None:
# 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
lower_lip_order = [
65,
66,
62,
70,
69,
18,
19,
20,
21,
22,
23,
24,
0,
8,
7,
6,
5,
4,
3,
2,
65,
]
lower_lip_landmarks = landmarks[lower_lip_order].astype(
np.float32
) # Use float for precise calculations
# Calculate the center of the landmarks
center = np.mean(lower_lip_landmarks, axis=0)
# Expand the landmarks outward
expansion_factor = (
1 + modules.globals.mask_down_size
) # Adjust this for more or less expansion
expanded_landmarks = (lower_lip_landmarks - center) * expansion_factor + center
# Extend the top lip part
toplip_indices = [
20,
0,
1,
2,
3,
4,
5,
] # Indices for landmarks 2, 65, 66, 62, 70, 69, 18
toplip_extension = (
modules.globals.mask_size * 0.5
) # Adjust this factor to control the extension
for idx in toplip_indices:
direction = expanded_landmarks[idx] - center
direction = direction / np.linalg.norm(direction)
expanded_landmarks[idx] += direction * toplip_extension
# Extend the bottom part (chin area)
chin_indices = [
11,
12,
13,
14,
15,
16,
] # Indices for landmarks 21, 22, 23, 24, 0, 8
chin_extension = 2 * 0.2 # Adjust this factor to control the extension
for idx in chin_indices:
expanded_landmarks[idx][1] += (
expanded_landmarks[idx][1] - center[1]
) * chin_extension
# Convert back to integer coordinates
expanded_landmarks = expanded_landmarks.astype(np.int32)
# Calculate bounding box for the expanded lower mouth
min_x, min_y = np.min(expanded_landmarks, axis=0)
max_x, max_y = np.max(expanded_landmarks, axis=0)
# Add some padding to the bounding box
padding = int((max_x - min_x) * 0.1) # 10% padding
min_x = max(0, min_x - padding)
min_y = max(0, min_y - padding)
max_x = min(frame.shape[1], max_x + padding)
max_y = min(frame.shape[0], max_y + padding)
# Ensure the bounding box dimensions are valid
if max_x <= min_x or max_y <= min_y:
if (max_x - min_x) <= 1:
max_x = min_x + 1
if (max_y - min_y) <= 1:
max_y = min_y + 1
# Create the mask
mask_roi = np.zeros((max_y - min_y, max_x - min_x), dtype=np.uint8)
cv2.fillPoly(mask_roi, [expanded_landmarks - [min_x, min_y]], 255)
# Apply Gaussian blur to soften the mask edges
mask_roi = cv2.GaussianBlur(mask_roi, (15, 15), 5)
# Place the mask ROI in the full-sized mask
mask[min_y:max_y, min_x:max_x] = mask_roi
# Extract the masked area from the frame
mouth_cutout = frame[min_y:max_y, min_x:max_x].copy()
# Return the expanded lower lip polygon in original frame coordinates
lower_lip_polygon = expanded_landmarks
return mask, mouth_cutout, (min_x, min_y, max_x, max_y), lower_lip_polygon
def draw_mouth_mask_visualization(
frame: Frame, face: Face, mouth_mask_data: tuple
) -> Frame:
landmarks = face.landmark_2d_106
if landmarks is not None and mouth_mask_data is not None:
mask, mouth_cutout, (min_x, min_y, max_x, max_y), lower_lip_polygon = (
mouth_mask_data
)
vis_frame = frame.copy()
# Ensure coordinates are within frame bounds
height, width = vis_frame.shape[:2]
min_x, min_y = max(0, min_x), max(0, min_y)
max_x, max_y = min(width, max_x), min(height, max_y)
# Adjust mask to match the region size
mask_region = mask[0 : max_y - min_y, 0 : max_x - min_x]
# Remove the color mask overlay
# color_mask = cv2.applyColorMap((mask_region * 255).astype(np.uint8), cv2.COLORMAP_JET)
# Ensure shapes match before blending
vis_region = vis_frame[min_y:max_y, min_x:max_x]
# Remove blending with color_mask
# if vis_region.shape[:2] == color_mask.shape[:2]:
# blended = cv2.addWeighted(vis_region, 0.7, color_mask, 0.3, 0)
# vis_frame[min_y:max_y, min_x:max_x] = blended
# Draw the lower lip polygon
cv2.polylines(vis_frame, [lower_lip_polygon], True, (0, 255, 0), 2)
# Remove the red box
# cv2.rectangle(vis_frame, (min_x, min_y), (max_x, max_y), (0, 0, 255), 2)
# Visualize the feathered mask
feather_amount = max(
1,
min(
30,
(max_x - min_x) // modules.globals.mask_feather_ratio,
(max_y - min_y) // modules.globals.mask_feather_ratio,
),
)
# Ensure kernel size is odd
kernel_size = 2 * feather_amount + 1
feathered_mask = cv2.GaussianBlur(
mask_region.astype(float), (kernel_size, kernel_size), 0
)
feathered_mask = (feathered_mask / feathered_mask.max() * 255).astype(np.uint8)
# Remove the feathered mask color overlay
# color_feathered_mask = cv2.applyColorMap(feathered_mask, cv2.COLORMAP_VIRIDIS)
# Ensure shapes match before blending feathered mask
# if vis_region.shape == color_feathered_mask.shape:
# blended_feathered = cv2.addWeighted(vis_region, 0.7, color_feathered_mask, 0.3, 0)
# vis_frame[min_y:max_y, min_x:max_x] = blended_feathered
# Add labels
cv2.putText(
vis_frame,
"Lower Mouth Mask",
(min_x, min_y - 10),
cv2.FONT_HERSHEY_SIMPLEX,
0.5,
(255, 255, 255),
1,
)
cv2.putText(
vis_frame,
"Feathered Mask",
(min_x, max_y + 20),
cv2.FONT_HERSHEY_SIMPLEX,
0.5,
(255, 255, 255),
1,
)
return vis_frame
return frame
def apply_mouth_area(
frame: np.ndarray,
mouth_cutout: np.ndarray,
mouth_box: tuple,
face_mask: np.ndarray,
mouth_polygon: np.ndarray,
) -> np.ndarray:
min_x, min_y, max_x, max_y = mouth_box
box_width = max_x - min_x
box_height = max_y - min_y
if (
mouth_cutout is None
or box_width is None
or box_height is None
or face_mask is None
or mouth_polygon is None
):
return frame
try:
resized_mouth_cutout = cv2.resize(mouth_cutout, (box_width, box_height))
roi = frame[min_y:max_y, min_x:max_x]
if roi.shape != resized_mouth_cutout.shape:
resized_mouth_cutout = cv2.resize(
resized_mouth_cutout, (roi.shape[1], roi.shape[0])
)
color_corrected_mouth = apply_color_transfer(resized_mouth_cutout, roi)
# Use the provided mouth polygon to create the mask
polygon_mask = np.zeros(roi.shape[:2], dtype=np.uint8)
adjusted_polygon = mouth_polygon - [min_x, min_y]
cv2.fillPoly(polygon_mask, [adjusted_polygon], 255)
# Apply feathering to the polygon mask
feather_amount = min(
30,
box_width // modules.globals.mask_feather_ratio,
box_height // modules.globals.mask_feather_ratio,
)
feathered_mask = cv2.GaussianBlur(
polygon_mask.astype(float), (0, 0), feather_amount
)
feathered_mask = feathered_mask / feathered_mask.max()
face_mask_roi = face_mask[min_y:max_y, min_x:max_x]
combined_mask = feathered_mask * (face_mask_roi / 255.0)
combined_mask = combined_mask[:, :, np.newaxis]
blended = (
color_corrected_mouth * combined_mask + roi * (1 - combined_mask)
).astype(np.uint8)
# Apply face mask to blended result
face_mask_3channel = (
np.repeat(face_mask_roi[:, :, np.newaxis], 3, axis=2) / 255.0
)
final_blend = blended * face_mask_3channel + roi * (1 - face_mask_3channel)
frame[min_y:max_y, min_x:max_x] = final_blend.astype(np.uint8)
except Exception as e:
pass
return frame
def create_face_mask(face: Face, frame: Frame) -> np.ndarray:
mask = np.zeros(frame.shape[:2], dtype=np.uint8)
landmarks = face.landmark_2d_106
if landmarks is not None:
# Convert landmarks to int32
landmarks = landmarks.astype(np.int32)
# Extract facial features
right_side_face = landmarks[0:16]
left_side_face = landmarks[17:32]
right_eye = landmarks[33:42]
right_eye_brow = landmarks[43:51]
left_eye = landmarks[87:96]
left_eye_brow = landmarks[97:105]
# Calculate forehead extension
right_eyebrow_top = np.min(right_eye_brow[:, 1])
left_eyebrow_top = np.min(left_eye_brow[:, 1])
eyebrow_top = min(right_eyebrow_top, left_eyebrow_top)
face_top = np.min([right_side_face[0, 1], left_side_face[-1, 1]])
forehead_height = face_top - eyebrow_top
extended_forehead_height = int(forehead_height * 5.0) # Extend by 50%
# Create forehead points
forehead_left = right_side_face[0].copy()
forehead_right = left_side_face[-1].copy()
forehead_left[1] -= extended_forehead_height
forehead_right[1] -= extended_forehead_height
# Combine all points to create the face outline
face_outline = np.vstack(
[
[forehead_left],
right_side_face,
left_side_face[
::-1
], # Reverse left side to create a continuous outline
[forehead_right],
]
)
# Calculate padding
padding = int(
np.linalg.norm(right_side_face[0] - left_side_face[-1]) * 0.05
) # 5% of face width
# Create a slightly larger convex hull for padding
hull = cv2.convexHull(face_outline)
hull_padded = []
for point in hull:
x, y = point[0]
center = np.mean(face_outline, axis=0)
direction = np.array([x, y]) - center
direction = direction / np.linalg.norm(direction)
padded_point = np.array([x, y]) + direction * padding
hull_padded.append(padded_point)
hull_padded = np.array(hull_padded, dtype=np.int32)
# Fill the padded convex hull
cv2.fillConvexPoly(mask, hull_padded, 255)
# Smooth the mask edges
mask = cv2.GaussianBlur(mask, (5, 5), 3)
return mask
def apply_color_transfer(source, target):
"""
Apply color transfer from target to source image
"""
source = cv2.cvtColor(source, cv2.COLOR_BGR2LAB).astype("float32")
target = cv2.cvtColor(target, cv2.COLOR_BGR2LAB).astype("float32")
source_mean, source_std = cv2.meanStdDev(source)
target_mean, target_std = cv2.meanStdDev(target)
# Reshape mean and std to be broadcastable
source_mean = source_mean.reshape(1, 1, 3)
source_std = source_std.reshape(1, 1, 3)
target_mean = target_mean.reshape(1, 1, 3)
target_std = target_std.reshape(1, 1, 3)
# Perform the color transfer
source = (source - source_mean) * (target_std / source_std) + target_mean
return cv2.cvtColor(np.clip(source, 0, 255).astype("uint8"), cv2.COLOR_LAB2BGR)
+45 -20
View File
@@ -27,6 +27,7 @@ from modules.utilities import (
)
from modules.video_capture import VideoCapturer
from modules.gettext import LanguageManager
from modules import globals
import platform
if platform.system() == "Windows":
@@ -160,12 +161,12 @@ def create_root(start: Callable[[], None], destroy: Callable[[], None]) -> ctk.C
select_face_button = ctk.CTkButton(
root, text=_("Select a face"), cursor="hand2", command=lambda: select_source_path()
)
select_face_button.place(relx=0.1, rely=0.4, relwidth=0.3, relheight=0.1)
select_face_button.place(relx=0.1, rely=0.375, relwidth=0.3, relheight=0.1)
swap_faces_button = ctk.CTkButton(
root, text="", cursor="hand2", command=lambda: swap_faces_paths()
)
swap_faces_button.place(relx=0.45, rely=0.4, relwidth=0.1, relheight=0.1)
swap_faces_button.place(relx=0.45, rely=0.375, relwidth=0.1, relheight=0.1)
select_target_button = ctk.CTkButton(
root,
@@ -173,7 +174,35 @@ def create_root(start: Callable[[], None], destroy: Callable[[], None]) -> ctk.C
cursor="hand2",
command=lambda: select_target_path(),
)
select_target_button.place(relx=0.6, rely=0.4, relwidth=0.3, relheight=0.1)
select_target_button.place(relx=0.6, rely=0.375, relwidth=0.3, relheight=0.1)
transparency_values = ["0%","25%", "50%", "75%", "100%"]
transparency_var = ctk.StringVar(value="100%") # Default to 100%
def on_transparency_change(value: str):
percentage = int(value.strip('%'))
modules.globals.opacity = percentage / 100.0
if percentage == 0:
modules.globals.fp_ui["face_enhancer"] = False
update_status("Transparency set to 0% - Face swapping disabled.")
elif percentage == 100:
modules.globals.face_swapper_enabled = True
update_status("Transparency set to 100%.")
else:
modules.globals.face_swapper_enabled = True
update_status(f"Transparency set to {value}")
transparency_label = ctk.CTkLabel(root, text="Transparency:")
transparency_label.place(relx=0.1, rely=0.5, relwidth=0.2, relheight=0.05)
transparency_dropdown = ctk.CTkOptionMenu(
root,
values=transparency_values,
variable=transparency_var,
command=on_transparency_change,
)
transparency_dropdown.place(relx=0.35, rely=0.5, relwidth=0.25, relheight=0.05)
keep_fps_value = ctk.BooleanVar(value=modules.globals.keep_fps)
keep_fps_checkbox = ctk.CTkSwitch(
@@ -397,7 +426,7 @@ def analyze_target(start: Callable[[], None], root: ctk.CTk):
return
if modules.globals.map_faces:
modules.globals.source_target_map = []
modules.globals.souce_target_map = []
if is_image(modules.globals.target_path):
update_status("Getting unique faces")
@@ -406,8 +435,8 @@ def analyze_target(start: Callable[[], None], root: ctk.CTk):
update_status("Getting unique faces")
get_unique_faces_from_target_video()
if len(modules.globals.source_target_map) > 0:
create_source_target_popup(start, root, modules.globals.source_target_map)
if len(modules.globals.souce_target_map) > 0:
create_source_target_popup(start, root, modules.globals.souce_target_map)
else:
update_status("No faces found in target")
else:
@@ -696,21 +725,17 @@ def check_and_ignore_nsfw(target, destroy: Callable = None) -> bool:
def fit_image_to_size(image, width: int, height: int):
if width is None or height is None or width <= 0 or height <= 0:
if width is None and height is None:
return image
h, w, _ = image.shape
ratio_h = 0.0
ratio_w = 0.0
ratio_w = width / w
ratio_h = height / h
# Use the smaller ratio to ensure the image fits within the given dimensions
ratio = min(ratio_w, ratio_h)
# Compute new dimensions, ensuring they're at least 1 pixel
new_width = max(1, int(ratio * w))
new_height = max(1, int(ratio * h))
new_size = (new_width, new_height)
if width > height:
ratio_h = height / h
else:
ratio_w = width / w
ratio = max(ratio_w, ratio_h)
new_size = (int(ratio * w), int(ratio * h))
return cv2.resize(image, dsize=new_size)
@@ -791,9 +816,9 @@ def webcam_preview(root: ctk.CTk, camera_index: int):
return
create_webcam_preview(camera_index)
else:
modules.globals.source_target_map = []
modules.globals.souce_target_map = []
create_source_target_popup_for_webcam(
root, modules.globals.source_target_map, camera_index
root, modules.globals.souce_target_map, camera_index
)
@@ -1203,4 +1228,4 @@ def update_webcam_target(
target_label_dict_live[button_num] = target_image
else:
update_pop_live_status("Face could not be detected in last upload!")
return map
return map
+12 -11
View File
@@ -1,20 +1,21 @@
--extra-index-url https://download.pytorch.org/whl/cu118
numpy>=1.23.5,<2
typing-extensions>=4.8.0
opencv-python==4.11.0.86
onnx==1.17.0
cv2_enumerate_cameras==1.1.18.3
opencv-python==4.10.0.84
cv2_enumerate_cameras==1.1.15
onnx==1.16.0
insightface==0.7.3
psutil==5.9.8
tk==0.1.0
customtkinter==5.2.2
pillow==11.1.0
torch; sys_platform != 'darwin' --index-url https://download.pytorch.org/whl/cu126
torch; sys_platform == 'darwin' --index-url https://download.pytorch.org/whl/cu126
torchvision; sys_platform != 'darwin' --index-url https://download.pytorch.org/whl/cu126
torchvision; sys_platform == 'darwin' --index-url https://download.pytorch.org/whl/cu126
pillow==9.5.0
torch==2.0.1+cu118; sys_platform != 'darwin'
torch==2.0.1; sys_platform == 'darwin'
torchvision==0.15.2+cu118; sys_platform != 'darwin'
torchvision==0.15.2; sys_platform == 'darwin'
onnxruntime-silicon==1.16.3; sys_platform == 'darwin' and platform_machine == 'arm64'
onnxruntime-gpu==1.21; sys_platform != 'darwin'
tensorflow; sys_platform != 'darwin'
onnxruntime-gpu==1.16.3; sys_platform != 'darwin'
tensorflow==2.12.1; sys_platform != 'darwin'
opennsfw2==0.10.2
protobuf==4.23.2
tqdm==4.66.4