Compare commits

..

35 Commits

Author SHA1 Message Date
Kenneth Estanislao 61dae91439 Revert "Merge pull request #566 from pereiraroland26/main"
This reverts commit 5d450b4352.
2024-10-04 15:57:48 +08:00
Kenneth Estanislao 5d450b4352 Merge pull request #566 from pereiraroland26/main
Added support for multiple faces
2024-10-04 15:55:31 +08:00
Kenneth Estanislao a11ccf9c49 Merge pull request #621 from saleweaver/experimental
Significantly improve video resolution/quality using ESPCN_x4 model
2024-09-23 11:59:23 +08:00
Michael 59cc742197 Add option to change the scale factor.
Add readme information about super resolution.
2024-09-23 02:49:48 +01:00
Michael 2066560a95 Significantly improve video resolution/quality using ESPCN_x4 model 2024-09-23 02:48:59 +01:00
Michael 1af9abda2f Significantly improve video resolution/quality using ESPCN_x4 model 2024-09-23 02:25:14 +01:00
Kenneth Estanislao 91884eebf7 Merge pull request #615 from saleweaver/experimental
Adding headless parameter to arguments to run from the cli, reenabling macOS compatibility
2024-09-23 04:01:23 +08:00
Michael 4686716c59 add to README.md 2024-09-22 18:39:00 +01:00
Michael f4028d3949 Fix underscore/hyphen 2024-09-22 18:33:02 +01:00
Michael 07c735e9d2 Allows to set the upscale factor for gfpgan face_enhancer 2024-09-22 18:31:06 +01:00
Michael aa021b6aa0 better import condition 2024-09-22 18:11:02 +01:00
Michael 0e3805e200 added headless argument to readme 2024-09-22 17:57:46 +01:00
Michael 5cabbffda8 - removed unused import statements
- added macOS specific required library to requirements.txt
- conditional import of pygrabber, which is unavailable for macOS
2024-09-22 17:55:26 +01:00
Michael 0d4676591e - removed unused import statements
- added macOS specific required library to requirements.txt
- conditional import of pygrabber, which is unavailable for macOS
2024-09-22 17:54:44 +01:00
Michael c2cc885672 Adding headless parameter to arguments to run from the cli 2024-09-21 22:41:47 +01:00
Kenneth Estanislao e36c746c81 Update setup_deep_live_cam.bat 2024-09-08 20:31:36 +08:00
barongello 14ab470dcc Adding a swap faces button to easily swap source/target images 2024-08-27 12:44:47 +08:00
Kenneth Estanislao 4dc4746235 update inswapper 2024-08-21 14:40:15 +08:00
Kenneth Estanislao ac8feff652 Merge pull request #329 from bit-wrangler/experimental
Added virtual camera output and fetching of input camera devices with names using pygrabber on windows and linux
2024-08-16 00:58:50 +08:00
Aleksandr Spiridonov a90c4facc5 added a note to README to document new virtual cam output feature 2024-08-15 12:27:52 -04:00
Aleksandr Spiridonov 575373beac fixed variable names not matching after merge conflict resolution from upstream 2024-08-15 12:22:19 -04:00
Aleksandr Spiridonov b8cdad5cce Merge remote-tracking branch 'parent/experimental' into experimental 2024-08-15 12:15:53 -04:00
Kenneth Estanislao 137ac597ef Merge pull request #293 from vic4key/experimental
To fix bugs and support more options for the Live function (see details in Commits tab)
2024-08-15 13:44:53 +08:00
Aleksandr Spiridonov f976885456 updated rely coords for the taller window 2024-08-15 01:36:24 -04:00
Aleksandr Spiridonov cd2c3c2103 added virtual cam output 2024-08-15 01:31:10 -04:00
Aleksandr Spiridonov 3fbc1d0433 added virtual cam output toggle 2024-08-15 01:04:57 -04:00
Aleksandr Spiridonov b4cf8854f8 refactored camera preview to use a loop function 2024-08-15 00:50:14 -04:00
Aleksandr Spiridonov eb733ad8c5 started using pygrabber to get input cameras with names; fixed issue with webcam preview not stopping when the preview window is closed 2024-08-15 00:42:53 -04:00
Vic P c6c41b8d0d Support the following options:
- The live camera display as you see it in the front-facing camera frame (like iPhone's Mirror Front Camera).
- The live camera frame is resizable.
Note: These options are turned off by default. Enabling both options may reduce performance by ~2%.

Signed-off-by: Vic P <vic4key@gmail.com>
2024-08-15 02:25:29 +07:00
Vic P 55c8d8181c Fix an issue that the Live function where the camera was not released when the user closed the live window.
Signed-off-by: Vic P <vic4key@gmail.com>
2024-08-14 00:48:01 +07:00
Kenneth Estanislao 4ddcd60c49 Merge pull request #237 from vic4key/experimental
Fix & Improve the NSFW function
2024-08-13 12:10:14 +08:00
Vic P 408b0f4cf7 ## Fix & Improve the NSFW function
- Fixed incorrect state usage.
- Removed the redundant argument that caused exceptions.
- Prevented the app from closing when an image is flagged as NSFW.
2024-08-13 04:16:34 +07:00
Kenneth Estanislao 78c808ef36 Merge pull request #166 from zoharbabin/experimental
Refactor and Optimize Cross-Platform Support
2024-08-12 12:27:35 +08:00
Zohar Babin 6b0cc74957 Refactor and Optimize Cross-Platform Support, Error Handling, and UI Enhancements 2024-08-10 22:41:45 -04:00
Dmitry Samoylenko 8d3072d906 Enable to choose a camera device in UI
Signed-off-by: samoylenkodmitry <samoylenkodmitry@gmail.com>
2024-08-10 14:08:29 +08:00
42 changed files with 1203 additions and 3543 deletions
-26
View File
@@ -1,26 +0,0 @@
***[Remove this]The issue would be closed without notice and be considered spam if the template is not followed.***
**Describe the bug**
A clear and concise description of what the bug is.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Error Message**
`<The error message in terminal>`
**Desktop (please complete the following information):**
- OS: [e.g. Windows]
- Version [e.g. 22]
- GPU
- CPU
**Additional context**
Add any other context about the problem here.
**Confirmation (Mandatory)**
- [ ] I have followed the template
- [ ] This is not a query about how to increase performance
- [ ] I have checked the issues page, and this is not a duplicate
+1 -7
View File
@@ -6,23 +6,17 @@ __pycache__/
.todo
*.log
*.backup
tf_env/
*.png
*.mp4
*.mkv
.tmp/
temp/
.venv/
venv/
env/
workflow/
gfpgan/
models/inswapper_128.onnx
models/GFPGANv1.4.pth
*.onnx
models/DMDNet.pth
faceswap/
.vscode/
switch_states.json
venv.rar
+1
View File
@@ -0,0 +1 @@
3.10.14
-38
View File
@@ -1,38 +0,0 @@
# Collaboration Guidelines and Codebase Quality Standards
To ensure smooth collaboration and maintain the high quality of our codebase, please adhere to the following guidelines:
## Branching Strategy
* **`premain`**:
* Always push your changes to the `premain` branch initially.
* This safeguards the `main` branch from unintentional disruptions.
* All tests will be performed on the `premain` branch.
* Changes will only be merged into `main` after several hours or days of rigorous testing.
* **`experimental`**:
* For large or potentially disruptive changes, use the `experimental` branch.
* This allows for thorough discussion and review before considering a merge into `main`.
## Pre-Pull Request Checklist
Before creating a Pull Request (PR), ensure you have completed the following tests:
### Functionality
* **Realtime Faceswap**:
* Test with face enhancer **enabled** and **disabled**.
* **Map Faces**:
* Test with both options (**enabled** and **disabled**).
* **Camera Listing**:
* Verify that all cameras are listed accurately.
### Stability
* **Realtime FPS**:
* Confirm that there is no drop in real-time frames per second (FPS).
* **Boot Time**:
* Changes should not negatively impact the boot time of either the application or the real-time faceswap feature.
* **GPU Overloading**:
* Test for a minimum of 15 minutes to guarantee no GPU overloading, which could lead to crashes.
* **App Performance**:
* The application should remain responsive and not exhibit any lag.
+91 -284
View File
@@ -1,326 +1,163 @@
<h1 align="center">Deep-Live-Cam</h1>
<p align="center">
Real-time face swap and video deepfake with a single click and only a single image.
</p>
<p align="center">
<a href="https://trendshift.io/repositories/11395" target="_blank"><img src="https://trendshift.io/api/badge/repositories/11395" alt="hacksider%2FDeep-Live-Cam | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
</p>
<p align="center">
<img src="media/demo.gif" alt="Demo GIF" width="800">
</p>
## Disclaimer
This deepfake software is designed to be a productive tool for the AI-generated media industry. It can assist artists in animating custom characters, creating engaging content, and even using models for clothing design.
We are aware of the potential for unethical applications and are committed to preventative measures. A built-in check prevents the program from processing inappropriate media (nudity, graphic content, sensitive material like war footage, etc.). We will continue to develop this project responsibly, adhering to the law and ethics. We may shut down the project or add watermarks if legally required.
- Ethical Use: Users are expected to use this software responsibly and legally. If using a real person's face, obtain their consent and clearly label any output as a deepfake when sharing online.
- Content Restrictions: The software includes built-in checks to prevent processing inappropriate media, such as nudity, graphic content, or sensitive material.
- Legal Compliance: We adhere to all relevant laws and ethical guidelines. If legally required, we may shut down the project or add watermarks to the output.
- User Responsibility: We are not responsible for end-user actions. Users must ensure their use of the software aligns with ethical standards and legal requirements.
By using this software, you agree to these terms and commit to using it in a manner that respects the rights and dignity of others.
Users are expected to use this software responsibly and legally. If using a real person's face, obtain their consent and clearly label any output as a deepfake when sharing online. We are not responsible for end-user actions.
![demo-gif](demo.gif)
## Quick Start - Pre-built (Windows / Nvidia)
## Disclaimer
This software is meant to be a productive contribution to the rapidly growing AI-generated media industry. It will help artists with tasks such as animating a custom character or using the character as a model for clothing etc.
<a href="https://hacksider.gumroad.com/l/vccdmm"> <img src="https://github.com/user-attachments/assets/7d993b32-e3e8-4cd3-bbfb-a549152ebdd5" width="285" height="77" />
The developers of this software are aware of its possible unethical applications and are committed to take preventative measures against them. It has a built-in check which prevents the program from working on inappropriate media including but not limited to nudity, graphic content, sensitive material such as war footage etc. We will continue to develop this project in the positive direction while adhering to law and ethics. This project may be shut down or include watermarks on the output if requested by law.
##### This is the fastest build you can get if you have a discrete NVIDIA GPU.
###### These Pre-builts are perfect for non-technical users or those who don't have time to, or can't manually install all the requirements. Just a heads-up: this is an open-source project, so you can also install it manually.
Users of this software are expected to use this software responsibly while abiding the local law. If face of a real person is being used, users are suggested to get consent from the concerned person and clearly mention that it is a deepfake when posting content online. Developers of this software will not be responsible for actions of end-users.
## TLDR; Live Deepfake in just 3 Clicks
![easysteps](https://github.com/user-attachments/assets/af825228-852c-411b-b787-ffd9aac72fc6)
1. Select a face
2. Select which camera to use
3. Press live!
## How do I install it?
## Features & Uses - Everything is in real-time
### Mouth Mask
**Retain your original mouth for accurate movement using Mouth Mask**
<p align="center">
<img src="media/ludwig.gif" alt="resizable-gif">
</p>
### Face Mapping
**Use different faces on multiple subjects simultaneously**
<p align="center">
<img src="media/streamers.gif" alt="face_mapping_source">
</p>
### Your Movie, Your Face
**Watch movies with any face in real-time**
<p align="center">
<img src="media/movie.gif" alt="movie">
</p>
### Live Show
**Run Live shows and performances**
<p align="center">
<img src="media/live_show.gif" alt="show">
</p>
### Memes
**Create Your Most Viral Meme Yet**
<p align="center">
<img src="media/meme.gif" alt="show" width="450">
<br>
<sub>Created using Many Faces feature in Deep-Live-Cam</sub>
</p>
### Omegle
**Surprise people on Omegle**
<p align="center">
<video src="https://github.com/user-attachments/assets/2e9b9b82-fa04-4b70-9f56-b1f68e7672d0" width="450" controls></video>
</p>
## Installation (Manual)
**Please be aware that the installation requires technical skills and is not for beginners. Consider downloading the prebuilt version.**
<details>
<summary>Click to see the process</summary>
### Installation
This is more likely to work on your computer but will be slower as it utilizes the CPU.
**1. Set up Your Platform**
- Python (3.10 recommended)
### Basic: It is more likely to work on your computer but it will also be very slow. You can follow instructions for the basic install (This usually runs via **CPU**)
#### 1.Setup your platform
- python (3.10 recommended)
- pip
- git
- [ffmpeg](https://www.youtube.com/watch?v=OlNWCpFdVMA) - ```iex (irm ffmpeg.tc.ht)```
- [Visual Studio 2022 Runtimes (Windows)](https://visualstudio.microsoft.com/visual-cpp-build-tools/)
- [ffmpeg](https://www.youtube.com/watch?v=OlNWCpFdVMA)
- [visual studio 2022 runtimes (windows)](https://visualstudio.microsoft.com/visual-cpp-build-tools/)
#### 2. Clone Repository
https://github.com/hacksider/Deep-Live-Cam.git
**2. Clone the Repository**
#### 3. Download Models
```bash
git clone https://github.com/hacksider/Deep-Live-Cam.git
cd Deep-Live-Cam
1. [GFPGANv1.4](https://huggingface.co/hacksider/deep-live-cam/resolve/main/GFPGANv1.4.pth)
2. [inswapper_128_fp16.onnx](https://huggingface.co/hacksider/deep-live-cam/resolve/main/inswapper_128.onnx)
Then put those 2 files on the "**models**" folder
#### 4. Install dependency
We highly recommend to work with a `venv` to avoid issues.
```
**3. Download the Models**
1. [GFPGANv1.4](https://huggingface.co/hacksider/deep-live-cam/resolve/main/GFPGANv1.4.pth)
2. [inswapper\_128\_fp16.onnx](https://huggingface.co/hacksider/deep-live-cam/resolve/main/inswapper_128_fp16.onnx)
Place these files in the "**models**" folder.
**4. Install Dependencies**
We highly recommend using a `venv` to avoid issues.
For Windows:
```bash
python -m venv venv
venv\Scripts\activate
pip install -r requirements.txt
```
##### DONE!!! If you dont have any GPU, You should be able to run roop using `python run.py` command. Keep in mind that while running the program for first time, it will download some models which can take time depending on your network connection.
**For macOS:**
### *Proceed if you want to use GPU Acceleration
### CUDA Execution Provider (Nvidia)*
Apple Silicon (M1/M2/M3) requires specific setup:
1. Install [CUDA Toolkit 11.8](https://developer.nvidia.com/cuda-11-8-0-download-archive)
2. Install dependencies:
```bash
# Install Python 3.10 (specific version is important)
brew install python@3.10
# Install tkinter package (required for the GUI)
brew install python-tk@3.10
# Create and activate virtual environment with Python 3.10
python3.10 -m venv venv
source venv/bin/activate
# Install dependencies
pip install -r requirements.txt
```
** In case something goes wrong and you need to reinstall the virtual environment **
```bash
# Deactivate the virtual environment
rm -rf venv
# Reinstall the virtual environment
python -m venv venv
source venv/bin/activate
# install the dependencies again
pip install -r requirements.txt
```
**Run:** If you don't have a GPU, you can run Deep-Live-Cam using `python run.py`. Note that initial execution will download models (~300MB).
### GPU Acceleration
**CUDA Execution Provider (Nvidia)**
1. Install [CUDA Toolkit 11.8.0](https://developer.nvidia.com/cuda-11-8-0-download-archive)
2. Install dependencies:
```bash
pip uninstall onnxruntime onnxruntime-gpu
pip install onnxruntime-gpu==1.16.3
```
3. Usage:
3. Usage in case the provider is available:
```bash
```
python run.py --execution-provider cuda
```
**CoreML Execution Provider (Apple Silicon)**
### [](https://github.com/s0md3v/roop/wiki/2.-Acceleration#coreml-execution-provider-apple-silicon)CoreML Execution Provider (Apple Silicon)
Apple Silicon (M1/M2/M3) specific installation:
1. Install dependencies:
1. Make sure you've completed the macOS setup above using Python 3.10.
2. Install dependencies:
```bash
```
pip uninstall onnxruntime onnxruntime-silicon
pip install onnxruntime-silicon==1.13.1
```
3. Usage (important: specify Python 3.10):
2. Usage in case the provider is available:
```
python run.py --execution-provider coreml
```bash
python3.10 run.py --execution-provider coreml
```
**Important Notes for macOS:**
- You **must** use Python 3.10, not newer versions like 3.11 or 3.13
- Always run with `python3.10` command not just `python` if you have multiple Python versions installed
- If you get error about `_tkinter` missing, reinstall the tkinter package: `brew reinstall python-tk@3.10`
- If you get model loading errors, check that your models are in the correct folder
- If you encounter conflicts with other Python versions, consider uninstalling them:
```bash
# List all installed Python versions
brew list | grep python
# Uninstall conflicting versions if needed
brew uninstall --ignore-dependencies python@3.11 python@3.13
# Keep only Python 3.10
brew cleanup
```
### [](https://github.com/s0md3v/roop/wiki/2.-Acceleration#coreml-execution-provider-apple-legacy)CoreML Execution Provider (Apple Legacy)
**CoreML Execution Provider (Apple Legacy)**
1. Install dependencies:
1. Install dependencies:
```bash
```
pip uninstall onnxruntime onnxruntime-coreml
pip install onnxruntime-coreml==1.13.1
```
2. Usage:
2. Usage in case the provider is available:
```bash
```
python run.py --execution-provider coreml
```
**DirectML Execution Provider (Windows)**
### [](https://github.com/s0md3v/roop/wiki/2.-Acceleration#directml-execution-provider-windows)DirectML Execution Provider (Windows)
1. Install dependencies:
1. Install dependencies:
```bash
```
pip uninstall onnxruntime onnxruntime-directml
pip install onnxruntime-directml==1.15.1
```
2. Usage:
2. Usage in case the provider is available:
```bash
```
python run.py --execution-provider directml
```
**OpenVINO™ Execution Provider (Intel)**
### [](https://github.com/s0md3v/roop/wiki/2.-Acceleration#openvino-execution-provider-intel)OpenVINO™ Execution Provider (Intel)
1. Install dependencies:
1. Install dependencies:
```bash
```
pip uninstall onnxruntime onnxruntime-openvino
pip install onnxruntime-openvino==1.15.0
```
2. Usage:
2. Usage in case the provider is available:
```bash
```
python run.py --execution-provider openvino
```
</details>
## Usage
## How do I use it?
> Note: When you run this program for the first time, it will download some models ~300MB in size.
**1. Image/Video Mode**
Executing `python run.py` command will launch this window:
![gui-demo](instruction.png)
- Execute `python run.py`.
- Choose a source face image and a target image/video.
- Click "Start".
- The output will be saved in a directory named after the target video.
Choose a face (image with desired face) and the target image/video (image/video in which you want to replace the face) and click on `Start`. Open file explorer and navigate to the directory you select your output to be in. You will find a directory named `<video_title>` where you can see the frames being swapped in realtime. Once the processing is done, it will create the output file. That's it.
**2. Webcam Mode**
## For the webcam mode
Just follow the clicks on the screenshot
1. Select a face
2. Click live
3. Wait for a few second (it takes a longer time, usually 10 to 30 seconds before the preview shows up)
- Execute `python run.py`.
- Select a source face image.
- Click "Live".
- Wait for the preview to appear (10-30 seconds).
- Use a screen capture tool like OBS to stream.
- To change the face, select a new source image.
![demo-gif](demo.gif)
## Tips and Tricks
Just use your favorite screencapture to stream like OBS
> Note: In case you want to change your face, just select another picture, the preview mode will then restart (so just wait a bit).
Check out these helpful guides to get the most out of Deep-Live-Cam:
You can now use the virtual camera output (uses pyvirtualcam) by turning on the `Virtual Cam Output (OBS)` toggle which should output to the OBS Virtual Camera. Note: this may not work on macOS. You will get a preview as before, but now you will also have a virtual camera output which can be used in applications like Zoom.
- [Unlocking the Secrets to the Perfect Deepfake Image](https://deeplivecam.net/index.php/blog/tips-and-tricks/unlocking-the-secrets-to-the-perfect-deepfake-image) - Learn how to create the best deepfake with full head coverage
- [Video Call with DeepLiveCam](https://deeplivecam.net/index.php/blog/tips-and-tricks/video-call-with-deeplivecam) - Make your meetings livelier by using DeepLiveCam with OBS and meeting software
- [Have a Special Guest!](https://deeplivecam.net/index.php/blog/tips-and-tricks/have-a-special-guest) - Tutorial on how to use face mapping to add special guests to your stream
- [Watch Deepfake Movies in Realtime](https://deeplivecam.net/index.php/blog/tips-and-tricks/watch-deepfake-movies-in-realtime) - See yourself star in any video without processing the video
- [Better Quality without Sacrificing Speed](https://deeplivecam.net/index.php/blog/tips-and-tricks/better-quality-without-sacrificing-speed) - Tips for achieving better results without impacting performance
- [Instant Vtuber!](https://deeplivecam.net/index.php/blog/tips-and-tricks/instant-vtuber) - Create a new persona/vtuber easily using Metahuman Creator
Additional command line arguments are given below. To learn out what they do, check [this guide](https://github.com/s0md3v/roop/wiki/Advanced-Options).
Visit our [official blog](https://deeplivecam.net/index.php/blog/tips-and-tricks) for more tips and tutorials.
## Command Line Arguments (Unmaintained)
```
options:
-h, --help show this help message and exit
-s SOURCE_PATH, --source SOURCE_PATH select a source image
-t TARGET_PATH, --target TARGET_PATH select a target image or video
-s SOURCE_PATH, --source SOURCE_PATH select an source image
-t TARGET_PATH, --target TARGET_PATH select an target image or video
-o OUTPUT_PATH, --output OUTPUT_PATH select output file or directory
--frame-processor FRAME_PROCESSOR [FRAME_PROCESSOR ...] frame processors (choices: face_swapper, face_enhancer, ...)
--frame-processor FRAME_PROCESSOR [FRAME_PROCESSOR ...] frame processors (choices: face_swapper, face_enhancer, super_resolution...)
--keep-fps keep original fps
--keep-audio keep original audio
--keep-frames keep temporary frames
--many-faces process every face
--map-faces map source target faces
--mouth-mask mask the mouth region
--video-encoder {libx264,libx265,libvpx-vp9} adjust output video encoder
--video-quality [0-51] adjust output video quality
--live-mirror the live camera display as you see it in the front-facing camera frame
@@ -328,54 +165,24 @@ options:
--max-memory MAX_MEMORY maximum amount of RAM in GB
--execution-provider {cpu} [{cpu} ...] available execution provider (choices: cpu, ...)
--execution-threads EXECUTION_THREADS number of execution threads
--headless run in headless mode
--enhancer-upscale-factor Sets the upscale factor for the enhancer. Only applies if `face_enhancer` is set as a frame-processor
--source-image-scaling-factor Set the upscale factor for source images. Only applies if `face_swapper` is set as a frame-processor
-r SCALE, --super-resolution-scale-factor SCALE Super resolution scale factor, choices are 2, 3, 4
-v, --version show program's version number and exit
```
Looking for a CLI mode? Using the -s/--source argument will make the run program in cli mode.
## Press
To improve the video quality, you can use the `super_resolution` frame processor after swapping the faces. It will enhance the video quality by 2x, 3x or 4x. You can set the upscale factor using the `-r` or `--super-resolution-scale-factor` argument.
Processing time will increase with the upscale factor, but it's quite quick.
**We are always open to criticism and are ready to improve, that's why we didn't cherry-pick anything.**
- [*"Deep-Live-Cam goes viral, allowing anyone to become a digital doppelganger"*](https://arstechnica.com/information-technology/2024/08/new-ai-tool-enables-real-time-face-swapping-on-webcams-raising-fraud-concerns/) - Ars Technica
- [*"Thanks Deep Live Cam, shapeshifters are among us now"*](https://dataconomy.com/2024/08/15/what-is-deep-live-cam-github-deepfake/) - Dataconomy
- [*"This free AI tool lets you become anyone during video-calls"*](https://www.newsbytesapp.com/news/science/deep-live-cam-ai-impersonation-tool-goes-viral/story) - NewsBytes
- [*"OK, this viral AI live stream software is truly terrifying"*](https://www.creativebloq.com/ai/ok-this-viral-ai-live-stream-software-is-truly-terrifying) - Creative Bloq
- [*"Deepfake AI Tool Lets You Become Anyone in a Video Call With Single Photo"*](https://petapixel.com/2024/08/14/deep-live-cam-deepfake-ai-tool-lets-you-become-anyone-in-a-video-call-with-single-photo-mark-zuckerberg-jd-vance-elon-musk/) - PetaPixel
- [*"Deep-Live-Cam Uses AI to Transform Your Face in Real-Time, Celebrities Included"*](https://www.techeblog.com/deep-live-cam-ai-transform-face/) - TechEBlog
- [*"An AI tool that "makes you look like anyone" during a video call is going viral online"*](https://telegrafi.com/en/a-tool-that-makes-you-look-like-anyone-during-a-video-call-is-going-viral-on-the-Internet/) - Telegrafi
- [*"This Deepfake Tool Turning Images Into Livestreams is Topping the GitHub Charts"*](https://decrypt.co/244565/this-deepfake-tool-turning-images-into-livestreams-is-topping-the-github-charts) - Emerge
- [*"New Real-Time Face-Swapping AI Allows Anyone to Mimic Famous Faces"*](https://www.digitalmusicnews.com/2024/08/15/face-swapping-ai-real-time-mimic/) - Digital Music News
- [*"This real-time webcam deepfake tool raises alarms about the future of identity theft"*](https://www.diyphotography.net/this-real-time-webcam-deepfake-tool-raises-alarms-about-the-future-of-identity-theft/) - DIYPhotography
- [*"That's Crazy, Oh God. That's Fucking Freaky Dude... That's So Wild Dude"*](https://www.youtube.com/watch?time_continue=1074&v=py4Tc-Y8BcY) - SomeOrdinaryGamers
- [*"Alright look look look, now look chat, we can do any face we want to look like chat"*](https://www.youtube.com/live/mFsCe7AIxq8?feature=shared&t=2686) - IShowSpeed
```
## Credits
- [ffmpeg](https://ffmpeg.org/): for making video-related operations easy
- [deepinsight](https://github.com/deepinsight): for their [insightface](https://github.com/deepinsight/insightface) project which provided a well-made library and models. Please be reminded that the [use of the model is for non-commercial research purposes only](https://github.com/deepinsight/insightface?tab=readme-ov-file#license).
- [havok2-htwo](https://github.com/havok2-htwo): for sharing the code for webcam
- [GosuDRM](https://github.com/GosuDRM): for the open version of roop
- [pereiraroland26](https://github.com/pereiraroland26): Multiple faces support
- [vic4key](https://github.com/vic4key): For supporting/contributing to this project
- [kier007](https://github.com/kier007): for improving the user experience
- [qitianai](https://github.com/qitianai): for multi-lingual support
- and [all developers](https://github.com/hacksider/Deep-Live-Cam/graphs/contributors) behind libraries used in this project.
- Footnote: Please be informed that the base author of the code is [s0md3v](https://github.com/s0md3v/roop)
- All the wonderful users who helped make this project go viral by starring the repo ❤️
[![Stargazers](https://reporoster.com/stars/hacksider/Deep-Live-Cam)](https://github.com/hacksider/Deep-Live-Cam/stargazers)
## Contributions
![Alt](https://repobeats.axiom.co/api/embed/fec8e29c45dfdb9c5916f3a7830e1249308d20e1.svg "Repobeats analytics image")
## Stars to the Moon 🚀
<a href="https://star-history.com/#hacksider/deep-live-cam&Date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=hacksider/deep-live-cam&type=Date&theme=dark" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=hacksider/deep-live-cam&type=Date" />
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=hacksider/deep-live-cam&type=Date" />
</picture>
</a>
- [henryruhs](https://github.com/henryruhs): for being an irreplaceable contributor to the project
- [ffmpeg](https://ffmpeg.org/): for making video related operations easy
- [deepinsight](https://github.com/deepinsight): for their [insightface](https://github.com/deepinsight/insightface) project which provided a well-made library and models.
- [havok2-htwo](https://github.com/havok2-htwo) : for sharing the code for webcam
- [GosuDRM](https://github.com/GosuDRM/nsfw-roop) : for uncensoring roop
- and all developers behind libraries used in this project.
View File

Before

Width:  |  Height:  |  Size: 11 MiB

After

Width:  |  Height:  |  Size: 11 MiB

BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 6.2 MiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 80 KiB

Before

Width:  |  Height:  |  Size: 73 KiB

After

Width:  |  Height:  |  Size: 73 KiB

-46
View File
@@ -1,46 +0,0 @@
{
"Source x Target Mapper": "Source x Target Mapper",
"select an source image": "选择一个源图像",
"Preview": "预览",
"select an target image or video": "选择一个目标图像或视频",
"save image output file": "保存图像输出文件",
"save video output file": "保存视频输出文件",
"select an target image": "选择一个目标图像",
"source": "源",
"Select a target": "选择一个目标",
"Select a face": "选择一张脸",
"Keep audio": "保留音频",
"Face Enhancer": "面纹增强器",
"Many faces": "多脸",
"Show FPS": "显示帧率",
"Keep fps": "保持帧率",
"Keep frames": "保持帧数",
"Fix Blueish Cam": "修复偏蓝的摄像头",
"Mouth Mask": "口罩",
"Show Mouth Mask Box": "显示口罩盒",
"Start": "开始",
"Live": "直播",
"Destroy": "结束",
"Map faces": "识别人脸",
"Processing...": "处理中...",
"Processing succeed!": "处理成功!",
"Processing ignored!": "处理被忽略!",
"Failed to start camera": "启动相机失败",
"Please complete pop-up or close it.": "请先完成弹出窗口或者关闭它",
"Getting unique faces": "获取独特面部",
"Please select a source image first": "请先选择一个源图像",
"No faces found in target": "目标图像中没有人脸",
"Add": "添加",
"Clear": "清除",
"Submit": "确认",
"Select source image": "请选取源图像",
"Select target image": "请选取目标图像",
"Please provide mapping!": "请提供映射",
"Atleast 1 source with target is required!": "至少需要一个来源图像与目标图像相关!",
"At least 1 source with target is required!": "至少需要一个来源图像与目标图像相关!",
"Face could not be detected in last upload!": "最近上传的图像中没有检测到人脸!",
"Select Camera:": "选择摄像头",
"All mappings cleared!": "所有映射均已清除!",
"Mappings successfully submitted!": "成功提交映射!",
"Source x Target Mapper is already open.": "源 x 目标映射器已打开。"
}
Binary file not shown.

Before

Width:  |  Height:  |  Size: 5.2 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.8 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 9.0 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 8.2 MiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 5.3 MiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 5.0 MiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 14 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 13 MiB

+1 -4
View File
@@ -1,4 +1 @@
just put the models in this folder -
https://huggingface.co/hacksider/deep-live-cam/resolve/main/inswapper_128_fp16.onnx?download=true
https://github.com/TencentARC/GFPGAN/releases/download/v1.3.4/GFPGANv1.4.pth
just put the models in this folder
+27 -21
View File
@@ -1,32 +1,38 @@
from typing import Any
from typing import Any, Optional
import cv2
import modules.globals # Import the globals to check the color correction toggle
def get_video_frame(video_path: str, frame_number: int = 0) -> Any:
def get_video_frame(video_path: str, frame_number: int = 0) -> Optional[Any]:
"""Retrieve a specific frame from a video."""
capture = cv2.VideoCapture(video_path)
# Set MJPEG format to ensure correct color space handling
capture.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc(*'MJPG'))
# Only force RGB conversion if color correction is enabled
if modules.globals.color_correction:
capture.set(cv2.CAP_PROP_CONVERT_RGB, 1)
frame_total = capture.get(cv2.CAP_PROP_FRAME_COUNT)
capture.set(cv2.CAP_PROP_POS_FRAMES, min(frame_total, frame_number - 1))
if not capture.isOpened():
print(f"Error: Cannot open video file {video_path}")
return None
frame_total = int(capture.get(cv2.CAP_PROP_FRAME_COUNT))
# Ensure frame_number is within the valid range
frame_number = max(0, min(frame_number, frame_total - 1))
capture.set(cv2.CAP_PROP_POS_FRAMES, frame_number)
has_frame, frame = capture.read()
if has_frame and modules.globals.color_correction:
# Convert the frame color if necessary
frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
capture.release()
return frame if has_frame else None
if not has_frame:
print(f"Error: Cannot read frame {frame_number} from {video_path}")
return None
return frame
def get_video_frame_total(video_path: str) -> int:
"""Get the total number of frames in a video."""
capture = cv2.VideoCapture(video_path)
video_frame_total = int(capture.get(cv2.CAP_PROP_FRAME_COUNT))
if not capture.isOpened():
print(f"Error: Cannot open video file {video_path}")
return 0
frame_total = int(capture.get(cv2.CAP_PROP_FRAME_COUNT))
capture.release()
return video_frame_total
return frame_total
-32
View File
@@ -1,32 +0,0 @@
import numpy as np
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score
from typing import Any
def find_cluster_centroids(embeddings, max_k=10) -> Any:
inertia = []
cluster_centroids = []
K = range(1, max_k+1)
for k in K:
kmeans = KMeans(n_clusters=k, random_state=0)
kmeans.fit(embeddings)
inertia.append(kmeans.inertia_)
cluster_centroids.append({"k": k, "centroids": kmeans.cluster_centers_})
diffs = [inertia[i] - inertia[i+1] for i in range(len(inertia)-1)]
optimal_centroids = cluster_centroids[diffs.index(max(diffs)) + 1]['centroids']
return optimal_centroids
def find_closest_centroid(centroids: list, normed_face_embedding) -> list:
try:
centroids = np.array(centroids)
normed_face_embedding = np.array(normed_face_embedding)
similarities = np.dot(centroids, normed_face_embedding)
closest_centroid_index = np.argmax(similarities)
return closest_centroid_index, centroids[closest_centroid_index]
except ValueError:
return None
+235 -866
View File
File diff suppressed because it is too large Load Diff
+14 -176
View File
@@ -1,189 +1,27 @@
import os
import shutil
from typing import Any
from typing import Any, Optional
import insightface
import cv2
import numpy as np
import modules.globals
from tqdm import tqdm
from modules.typing import Frame
from modules.cluster_analysis import find_cluster_centroids, find_closest_centroid
from modules.utilities import get_temp_directory_path, create_temp, extract_frames, clean_temp, get_temp_frame_paths
from pathlib import Path
FACE_ANALYSER = None
FACE_ANALYSER: Optional[insightface.app.FaceAnalysis] = None
def get_face_analyser() -> Any:
def get_face_analyser() -> insightface.app.FaceAnalysis:
global FACE_ANALYSER
if FACE_ANALYSER is None:
FACE_ANALYSER = insightface.app.FaceAnalysis(name='buffalo_l', providers=modules.globals.execution_providers)
FACE_ANALYSER = insightface.app.FaceAnalysis(
name='buffalo_l',
providers=modules.globals.execution_providers
)
FACE_ANALYSER.prepare(ctx_id=0, det_size=(640, 640))
return FACE_ANALYSER
def get_one_face(frame: Frame) -> Optional[Any]:
faces = get_face_analyser().get(frame)
return min(faces, key=lambda x: x.bbox[0], default=None)
def get_one_face(frame: Frame) -> Any:
face = get_face_analyser().get(frame)
try:
return min(face, key=lambda x: x.bbox[0])
except ValueError:
return None
def get_many_faces(frame: Frame) -> Any:
try:
return get_face_analyser().get(frame)
except IndexError:
return None
def has_valid_map() -> bool:
for map in modules.globals.source_target_map:
if "source" in map and "target" in map:
return True
return False
def default_source_face() -> Any:
for map in modules.globals.source_target_map:
if "source" in map:
return map['source']['face']
return None
def simplify_maps() -> Any:
centroids = []
faces = []
for map in modules.globals.source_target_map:
if "source" in map and "target" in map:
centroids.append(map['target']['face'].normed_embedding)
faces.append(map['source']['face'])
modules.globals.simple_map = {'source_faces': faces, 'target_embeddings': centroids}
return None
def add_blank_map() -> Any:
try:
max_id = -1
if len(modules.globals.source_target_map) > 0:
max_id = max(modules.globals.source_target_map, key=lambda x: x['id'])['id']
modules.globals.source_target_map.append({
'id' : max_id + 1
})
except ValueError:
return None
def get_unique_faces_from_target_image() -> Any:
try:
modules.globals.source_target_map = []
target_frame = cv2.imread(modules.globals.target_path)
many_faces = get_many_faces(target_frame)
i = 0
for face in many_faces:
x_min, y_min, x_max, y_max = face['bbox']
modules.globals.source_target_map.append({
'id' : i,
'target' : {
'cv2' : target_frame[int(y_min):int(y_max), int(x_min):int(x_max)],
'face' : face
}
})
i = i + 1
except ValueError:
return None
def get_unique_faces_from_target_video() -> Any:
try:
modules.globals.source_target_map = []
frame_face_embeddings = []
face_embeddings = []
print('Creating temp resources...')
clean_temp(modules.globals.target_path)
create_temp(modules.globals.target_path)
print('Extracting frames...')
extract_frames(modules.globals.target_path)
temp_frame_paths = get_temp_frame_paths(modules.globals.target_path)
i = 0
for temp_frame_path in tqdm(temp_frame_paths, desc="Extracting face embeddings from frames"):
temp_frame = cv2.imread(temp_frame_path)
many_faces = get_many_faces(temp_frame)
for face in many_faces:
face_embeddings.append(face.normed_embedding)
frame_face_embeddings.append({'frame': i, 'faces': many_faces, 'location': temp_frame_path})
i += 1
centroids = find_cluster_centroids(face_embeddings)
for frame in frame_face_embeddings:
for face in frame['faces']:
closest_centroid_index, _ = find_closest_centroid(centroids, face.normed_embedding)
face['target_centroid'] = closest_centroid_index
for i in range(len(centroids)):
modules.globals.source_target_map.append({
'id' : i
})
temp = []
for frame in tqdm(frame_face_embeddings, desc=f"Mapping frame embeddings to centroids-{i}"):
temp.append({'frame': frame['frame'], 'faces': [face for face in frame['faces'] if face['target_centroid'] == i], 'location': frame['location']})
modules.globals.source_target_map[i]['target_faces_in_frame'] = temp
# dump_faces(centroids, frame_face_embeddings)
default_target_face()
except ValueError:
return None
def default_target_face():
for map in modules.globals.source_target_map:
best_face = None
best_frame = None
for frame in map['target_faces_in_frame']:
if len(frame['faces']) > 0:
best_face = frame['faces'][0]
best_frame = frame
break
for frame in map['target_faces_in_frame']:
for face in frame['faces']:
if face['det_score'] > best_face['det_score']:
best_face = face
best_frame = frame
x_min, y_min, x_max, y_max = best_face['bbox']
target_frame = cv2.imread(best_frame['location'])
map['target'] = {
'cv2' : target_frame[int(y_min):int(y_max), int(x_min):int(x_max)],
'face' : best_face
}
def dump_faces(centroids: Any, frame_face_embeddings: list):
temp_directory_path = get_temp_directory_path(modules.globals.target_path)
for i in range(len(centroids)):
if os.path.exists(temp_directory_path + f"/{i}") and os.path.isdir(temp_directory_path + f"/{i}"):
shutil.rmtree(temp_directory_path + f"/{i}")
Path(temp_directory_path + f"/{i}").mkdir(parents=True, exist_ok=True)
for frame in tqdm(frame_face_embeddings, desc=f"Copying faces to temp/./{i}"):
temp_frame = cv2.imread(frame['location'])
j = 0
for face in frame['faces']:
if face['target_centroid'] == i:
x_min, y_min, x_max, y_max = face['bbox']
if temp_frame[int(y_min):int(y_max), int(x_min):int(x_max)].size > 0:
cv2.imwrite(temp_directory_path + f"/{i}/{frame['frame']}_{j}.png", temp_frame[int(y_min):int(y_max), int(x_min):int(x_max)])
j += 1
def get_many_faces(frame: Frame) -> Optional[Any]:
faces = get_face_analyser().get(frame)
return faces if faces else None
-26
View File
@@ -1,26 +0,0 @@
import json
from pathlib import Path
class LanguageManager:
def __init__(self, default_language="en"):
self.current_language = default_language
self.translations = {}
self.load_language(default_language)
def load_language(self, language_code) -> bool:
"""load language file"""
if language_code == "en":
return True
try:
file_path = Path(__file__).parent.parent / f"locales/{language_code}.json"
with open(file_path, "r", encoding="utf-8") as file:
self.translations = json.load(file)
self.current_language = language_code
return True
except FileNotFoundError:
print(f"Language file not found: {language_code}")
return False
def _(self, key, default=None) -> str:
"""get translate text"""
return self.translations.get(key, default if default else key)
+16 -24
View File
@@ -1,43 +1,35 @@
import os
from typing import List, Dict, Any
from typing import List, Dict
ROOT_DIR = os.path.dirname(os.path.abspath(__file__))
WORKFLOW_DIR = os.path.join(ROOT_DIR, "workflow")
WORKFLOW_DIR = os.path.join(ROOT_DIR, 'workflow')
file_types = [
("Image", ("*.png", "*.jpg", "*.jpeg", "*.gif", "*.bmp")),
("Video", ("*.mp4", "*.mkv")),
('Image', ('*.png','*.jpg','*.jpeg','*.gif','*.bmp')),
('Video', ('*.mp4','*.mkv'))
]
source_target_map = []
simple_map = {}
source_path = None
target_path = None
output_path = None
frame_processors: List[str] = []
keep_fps = True
keep_audio = True
keep_frames = False
many_faces = False
map_faces = False
color_correction = False # New global variable for color correction toggle
nsfw_filter = False
keep_fps = None
keep_audio = None
keep_frames = None
many_faces = None
video_encoder = None
video_quality = None
live_mirror = False
live_resizable = True
live_mirror = None
live_resizable = None
max_memory = None
execution_providers: List[str] = []
execution_threads = None
headless = None
log_level = "error"
fp_ui: Dict[str, bool] = {"face_enhancer": False}
log_level = 'error'
fp_ui: Dict[str, bool] = {}
nsfw = None
camera_input_combobox = None
webcam_preview_running = False
show_fps = False
mouth_mask = False
show_mouth_mask_box = False
mask_feather_ratio = 8
mask_down_size = 0.50
mask_size = 1
enhancer_upscale_factor = 1
source_image_scaling_factor = 2
sr_scale_factor = 4
+3 -3
View File
@@ -1,3 +1,3 @@
name = 'Deep-Live-Cam'
version = '1.8'
edition = 'GitHub Edition'
name = 'Deep Live Cam'
version = '1.3.0'
edition = 'Portable'
+6 -16
View File
@@ -1,9 +1,6 @@
import numpy
import numpy as np
import opennsfw2
from PIL import Image
import cv2 # Add OpenCV import
import modules.globals # Import globals to access the color correction toggle
from modules.typing import Frame
MAX_PROBABILITY = 0.85
@@ -12,24 +9,17 @@ MAX_PROBABILITY = 0.85
model = None
def predict_frame(target_frame: Frame) -> bool:
# Convert the frame to RGB before processing if color correction is enabled
if modules.globals.color_correction:
target_frame = cv2.cvtColor(target_frame, cv2.COLOR_BGR2RGB)
global model
if model is None: model = opennsfw2.make_open_nsfw_model()
image = Image.fromarray(target_frame)
image = opennsfw2.preprocess_image(image, opennsfw2.Preprocessing.YAHOO)
global model
if model is None:
model = opennsfw2.make_open_nsfw_model()
views = numpy.expand_dims(image, axis=0)
views = np.expand_dims(image, axis=0)
_, probability = model.predict(views)[0]
return probability > MAX_PROBABILITY
def predict_image(target_path: str) -> bool:
return opennsfw2.predict_image(target_path) > MAX_PROBABILITY
probability = opennsfw2.predict_image(target_path)
return probability > MAX_PROBABILITY
def predict_video(target_path: str) -> bool:
_, probabilities = opennsfw2.predict_video_frames(video_path=target_path, frame_interval=100)
+25 -26
View File
@@ -17,57 +17,56 @@ FRAME_PROCESSORS_INTERFACE = [
'process_video'
]
def load_frame_processor_module(frame_processor: str) -> Any:
def load_frame_processor_module(frame_processor: str) -> ModuleType:
try:
frame_processor_module = importlib.import_module(f'modules.processors.frame.{frame_processor}')
# Ensure all required methods are present
for method_name in FRAME_PROCESSORS_INTERFACE:
if not hasattr(frame_processor_module, method_name):
sys.exit()
raise AttributeError(f"Missing required method {method_name} in {frame_processor} module.")
except ImportError:
print(f"Frame processor {frame_processor} not found")
sys.exit()
print(f"Error: Frame processor '{frame_processor}' not found.")
sys.exit(1)
except AttributeError as e:
print(e)
sys.exit(1)
return frame_processor_module
def get_frame_processors_modules(frame_processors: List[str]) -> List[ModuleType]:
global FRAME_PROCESSORS_MODULES
if not FRAME_PROCESSORS_MODULES:
for frame_processor in frame_processors:
frame_processor_module = load_frame_processor_module(frame_processor)
FRAME_PROCESSORS_MODULES.append(frame_processor_module)
FRAME_PROCESSORS_MODULES = [load_frame_processor_module(fp) for fp in frame_processors]
set_frame_processors_modules_from_ui(frame_processors)
return FRAME_PROCESSORS_MODULES
def set_frame_processors_modules_from_ui(frame_processors: List[str]) -> None:
global FRAME_PROCESSORS_MODULES
for frame_processor, state in modules.globals.fp_ui.items():
if state == True and frame_processor not in frame_processors:
frame_processor_module = load_frame_processor_module(frame_processor)
FRAME_PROCESSORS_MODULES.append(frame_processor_module)
if state and frame_processor not in frame_processors:
module = load_frame_processor_module(frame_processor)
FRAME_PROCESSORS_MODULES.append(module)
modules.globals.frame_processors.append(frame_processor)
if state == False:
try:
frame_processor_module = load_frame_processor_module(frame_processor)
FRAME_PROCESSORS_MODULES.remove(frame_processor_module)
modules.globals.frame_processors.remove(frame_processor)
except:
pass
elif not state and frame_processor in frame_processors:
module = load_frame_processor_module(frame_processor)
FRAME_PROCESSORS_MODULES.remove(module)
modules.globals.frame_processors.remove(frame_processor)
def multi_process_frame(source_path: str, temp_frame_paths: List[str], process_frames: Callable[[str, List[str], Any], None], progress: Any = None) -> None:
with ThreadPoolExecutor(max_workers=modules.globals.execution_threads) as executor:
futures = []
for path in temp_frame_paths:
future = executor.submit(process_frames, source_path, [path], progress)
futures.append(future)
futures = [executor.submit(process_frames, source_path, [path], progress) for path in temp_frame_paths]
for future in futures:
future.result()
def process_video(source_path: str, frame_paths: list[str], process_frames: Callable[[str, List[str], Any], None]) -> None:
def process_video(source_path: str, frame_paths: List[str], process_frames: Callable[[str, List[str], Any], None]) -> None:
progress_bar_format = '{l_bar}{bar}| {n_fmt}/{total_fmt} [{elapsed}<{remaining}, {rate_fmt}{postfix}]'
total = len(frame_paths)
with tqdm(total=total, desc='Processing', unit='frame', dynamic_ncols=True, bar_format=progress_bar_format) as progress:
progress.set_postfix({'execution_providers': modules.globals.execution_providers, 'execution_threads': modules.globals.execution_threads, 'max_memory': modules.globals.max_memory})
progress.set_postfix({
'execution_providers': modules.globals.execution_providers,
'execution_threads': modules.globals.execution_threads,
'max_memory': modules.globals.max_memory
})
multi_process_frame(source_path, frame_paths, process_frames, progress)
+17 -56
View File
@@ -8,82 +8,52 @@ import modules.globals
import modules.processors.frame.core
from modules.core import update_status
from modules.face_analyser import get_one_face
from modules.typing import Frame, Face
import platform
import torch
from modules.utilities import (
conditional_download,
is_image,
is_video,
)
from modules.typing import Frame, Face # Ensure these are imported
from modules.utilities import conditional_download, resolve_relative_path, is_image, is_video
FACE_ENHANCER = None
THREAD_SEMAPHORE = threading.Semaphore()
THREAD_LOCK = threading.Lock()
NAME = "DLC.FACE-ENHANCER"
abs_dir = os.path.dirname(os.path.abspath(__file__))
models_dir = os.path.join(
os.path.dirname(os.path.dirname(os.path.dirname(abs_dir))), "models"
)
NAME = 'DLC.FACE-ENHANCER'
def pre_check() -> bool:
download_directory_path = models_dir
conditional_download(
download_directory_path,
[
"https://github.com/TencentARC/GFPGAN/releases/download/v1.3.4/GFPGANv1.4.pth"
],
)
download_directory_path = resolve_relative_path('..\models')
conditional_download(download_directory_path, ['https://github.com/TencentARC/GFPGAN/releases/download/v1.3.4/GFPGANv1.4.pth'])
return True
def pre_start() -> bool:
if not is_image(modules.globals.target_path) and not is_video(
modules.globals.target_path
):
update_status("Select an image or video for target path.", NAME)
if not is_image(modules.globals.target_path) and not is_video(modules.globals.target_path):
update_status('Select an image or video for target path.', NAME)
return False
return True
def get_face_enhancer() -> Any:
global FACE_ENHANCER
with THREAD_LOCK:
if FACE_ENHANCER is None:
model_path = os.path.join(models_dir, "GFPGANv1.4.pth")
match platform.system():
case "Darwin": # Mac OS
if torch.backends.mps.is_available():
mps_device = torch.device("mps")
FACE_ENHANCER = gfpgan.GFPGANer(model_path=model_path, upscale=1, device=mps_device) # type: ignore[attr-defined]
else:
FACE_ENHANCER = gfpgan.GFPGANer(model_path=model_path, upscale=1) # type: ignore[attr-defined]
case _: # Other OS
FACE_ENHANCER = gfpgan.GFPGANer(model_path=model_path, upscale=1) # type: ignore[attr-defined]
model_path = resolve_relative_path('../models/GFPGANv1.4.pth')
FACE_ENHANCER = gfpgan.GFPGANer(
model_path=model_path,
upscale=modules.globals.enhancer_upscale_factor
) # type: ignore[attr-defined]
return FACE_ENHANCER
def enhance_face(temp_frame: Frame) -> Frame:
with THREAD_SEMAPHORE:
_, _, temp_frame = get_face_enhancer().enhance(temp_frame, paste_back=True)
_, _, temp_frame = get_face_enhancer().enhance(
temp_frame,
paste_back=True
)
return temp_frame
def process_frame(source_face: Face, temp_frame: Frame) -> Frame:
target_face = get_one_face(temp_frame)
if target_face:
temp_frame = enhance_face(temp_frame)
return temp_frame
def process_frames(
source_path: str, temp_frame_paths: List[str], progress: Any = None
) -> None:
def process_frames(source_path: str, temp_frame_paths: List[str], progress: Any = None) -> None:
for temp_frame_path in temp_frame_paths:
temp_frame = cv2.imread(temp_frame_path)
result = process_frame(None, temp_frame)
@@ -91,19 +61,10 @@ def process_frames(
if progress:
progress.update(1)
def process_image(source_path: str, target_path: str, output_path: str) -> None:
target_frame = cv2.imread(target_path)
result = process_frame(None, target_frame)
cv2.imwrite(output_path, result)
def process_video(source_path: str, temp_frame_paths: List[str]) -> None:
modules.processors.frame.core.process_video(None, temp_frame_paths, process_frames)
def process_frame_v2(temp_frame: Frame) -> Frame:
target_face = get_one_face(temp_frame)
if target_face:
temp_frame = enhance_face(temp_frame)
return temp_frame
+58 -576
View File
@@ -2,621 +2,103 @@ from typing import Any, List
import cv2
import insightface
import threading
import numpy as np
import os
import modules.globals
import logging
import modules.processors.frame.core
from modules.core import update_status
from modules.face_analyser import get_one_face, get_many_faces, default_source_face
from modules.face_analyser import get_one_face, get_many_faces
from modules.typing import Face, Frame
from modules.utilities import (
conditional_download,
is_image,
is_video,
)
from modules.cluster_analysis import find_closest_centroid
import os
from modules.utilities import conditional_download, resolve_relative_path, is_image, is_video
import numpy as np
FACE_SWAPPER = None
THREAD_LOCK = threading.Lock()
NAME = "DLC.FACE-SWAPPER"
abs_dir = os.path.dirname(os.path.abspath(__file__))
models_dir = os.path.join(
os.path.dirname(os.path.dirname(os.path.dirname(abs_dir))), "models"
)
NAME = 'DLC.FACE-SWAPPER'
def pre_check() -> bool:
download_directory_path = abs_dir
conditional_download(
download_directory_path,
[
"https://huggingface.co/hacksider/deep-live-cam/blob/main/inswapper_128_fp16.onnx"
],
)
download_directory_path = resolve_relative_path('../models')
conditional_download(download_directory_path, [
'https://huggingface.co/hacksider/deep-live-cam/blob/main/inswapper_128.onnx'
])
return True
def pre_start() -> bool:
if not modules.globals.map_faces and not is_image(modules.globals.source_path):
update_status("Select an image for source path.", NAME)
if not is_image(modules.globals.source_path):
update_status('Select an image for source path.', NAME)
return False
elif not modules.globals.map_faces and not get_one_face(
cv2.imread(modules.globals.source_path)
):
update_status("No face in source path detected.", NAME)
elif not get_one_face(cv2.imread(modules.globals.source_path)):
update_status('No face detected in the source path.', NAME)
return False
if not is_image(modules.globals.target_path) and not is_video(
modules.globals.target_path
):
update_status("Select an image or video for target path.", NAME)
if not is_image(modules.globals.target_path) and not is_video(modules.globals.target_path):
update_status('Select an image or video for target path.', NAME)
return False
return True
def get_face_swapper() -> Any:
global FACE_SWAPPER
with THREAD_LOCK:
if FACE_SWAPPER is None:
model_path = os.path.join(models_dir, "inswapper_128_fp16.onnx")
FACE_SWAPPER = insightface.model_zoo.get_model(
model_path, providers=modules.globals.execution_providers
)
model_path = resolve_relative_path('../models/inswapper_128.onnx')
FACE_SWAPPER = insightface.model_zoo.get_model(model_path, providers=modules.globals.execution_providers)
return FACE_SWAPPER
def upscale_image(image: np.ndarray, scaling_factor: int = modules.globals.source_image_scaling_factor) -> np.ndarray:
"""
Upscales the given image by the specified scaling factor.
Args:
image (np.ndarray): The input image to upscale.
scaling_factor (int): The factor by which to upscale the image.
Returns:
np.ndarray: The upscaled image.
"""
height, width = image.shape[:2]
new_size = (width * scaling_factor, height * scaling_factor)
upscaled_image = cv2.resize(image, new_size, interpolation=cv2.INTER_CUBIC)
return upscaled_image
def swap_face(source_face: Face, target_face: Face, temp_frame: Frame) -> Frame:
face_swapper = get_face_swapper()
# Apply the face swap
swapped_frame = face_swapper.get(
temp_frame, target_face, source_face, paste_back=True
)
if modules.globals.mouth_mask:
# Create a mask for the target face
face_mask = create_face_mask(target_face, temp_frame)
# Create the mouth mask
mouth_mask, mouth_cutout, mouth_box, lower_lip_polygon = (
create_lower_mouth_mask(target_face, temp_frame)
)
# Apply the mouth area
swapped_frame = apply_mouth_area(
swapped_frame, mouth_cutout, mouth_box, face_mask, lower_lip_polygon
)
if modules.globals.show_mouth_mask_box:
mouth_mask_data = (mouth_mask, mouth_cutout, mouth_box, lower_lip_polygon)
swapped_frame = draw_mouth_mask_visualization(
swapped_frame, target_face, mouth_mask_data
)
return swapped_frame
return get_face_swapper().get(temp_frame, target_face, source_face, paste_back=True)
def process_frame(source_face: Face, temp_frame: Frame) -> Frame:
if modules.globals.color_correction:
temp_frame = cv2.cvtColor(temp_frame, cv2.COLOR_BGR2RGB)
if modules.globals.many_faces:
many_faces = get_many_faces(temp_frame)
if many_faces:
for target_face in many_faces:
if source_face and target_face:
temp_frame = swap_face(source_face, target_face, temp_frame)
else:
print("Face detection failed for target/source.")
temp_frame = swap_face(source_face, target_face, temp_frame)
else:
target_face = get_one_face(temp_frame)
if target_face and source_face:
if target_face:
temp_frame = swap_face(source_face, target_face, temp_frame)
else:
logging.error("Face detection failed for target or source.")
return temp_frame
def process_frames(source_path: str, temp_frame_paths: List[str], progress: Any = None) -> None:
source_image = cv2.imread(source_path)
if source_image is None:
print(f"Failed to load source image from {source_path}")
return
# Upscale the source image for better quality
source_image_upscaled = upscale_image(source_image, scaling_factor=2)
source_face = get_one_face(source_image_upscaled)
def process_frame_v2(temp_frame: Frame, temp_frame_path: str = "") -> Frame:
if is_image(modules.globals.target_path):
if modules.globals.many_faces:
source_face = default_source_face()
for map in modules.globals.source_target_map:
target_face = map["target"]["face"]
temp_frame = swap_face(source_face, target_face, temp_frame)
elif not modules.globals.many_faces:
for map in modules.globals.source_target_map:
if "source" in map:
source_face = map["source"]["face"]
target_face = map["target"]["face"]
temp_frame = swap_face(source_face, target_face, temp_frame)
elif is_video(modules.globals.target_path):
if modules.globals.many_faces:
source_face = default_source_face()
for map in modules.globals.source_target_map:
target_frame = [
f
for f in map["target_faces_in_frame"]
if f["location"] == temp_frame_path
]
for frame in target_frame:
for target_face in frame["faces"]:
temp_frame = swap_face(source_face, target_face, temp_frame)
elif not modules.globals.many_faces:
for map in modules.globals.source_target_map:
if "source" in map:
target_frame = [
f
for f in map["target_faces_in_frame"]
if f["location"] == temp_frame_path
]
source_face = map["source"]["face"]
for frame in target_frame:
for target_face in frame["faces"]:
temp_frame = swap_face(source_face, target_face, temp_frame)
else:
detected_faces = get_many_faces(temp_frame)
if modules.globals.many_faces:
if detected_faces:
source_face = default_source_face()
for target_face in detected_faces:
temp_frame = swap_face(source_face, target_face, temp_frame)
elif not modules.globals.many_faces:
if detected_faces:
if len(detected_faces) <= len(
modules.globals.simple_map["target_embeddings"]
):
for detected_face in detected_faces:
closest_centroid_index, _ = find_closest_centroid(
modules.globals.simple_map["target_embeddings"],
detected_face.normed_embedding,
)
temp_frame = swap_face(
modules.globals.simple_map["source_faces"][
closest_centroid_index
],
detected_face,
temp_frame,
)
else:
detected_faces_centroids = []
for face in detected_faces:
detected_faces_centroids.append(face.normed_embedding)
i = 0
for target_embedding in modules.globals.simple_map[
"target_embeddings"
]:
closest_centroid_index, _ = find_closest_centroid(
detected_faces_centroids, target_embedding
)
temp_frame = swap_face(
modules.globals.simple_map["source_faces"][i],
detected_faces[closest_centroid_index],
temp_frame,
)
i += 1
return temp_frame
def process_frames(
source_path: str, temp_frame_paths: List[str], progress: Any = None
) -> None:
if not modules.globals.map_faces:
source_face = get_one_face(cv2.imread(source_path))
for temp_frame_path in temp_frame_paths:
temp_frame = cv2.imread(temp_frame_path)
try:
result = process_frame(source_face, temp_frame)
cv2.imwrite(temp_frame_path, result)
except Exception as exception:
print(exception)
pass
if progress:
progress.update(1)
else:
for temp_frame_path in temp_frame_paths:
temp_frame = cv2.imread(temp_frame_path)
try:
result = process_frame_v2(temp_frame, temp_frame_path)
cv2.imwrite(temp_frame_path, result)
except Exception as exception:
print(exception)
pass
if progress:
progress.update(1)
for temp_frame_path in temp_frame_paths:
temp_frame = cv2.imread(temp_frame_path)
try:
result = process_frame(source_face, temp_frame)
cv2.imwrite(temp_frame_path, result)
except Exception as exception:
print(f"Error processing frame {temp_frame_path}: {exception}")
if progress:
progress.update(1)
def process_image(source_path: str, target_path: str, output_path: str) -> None:
if not modules.globals.map_faces:
source_face = get_one_face(cv2.imread(source_path))
target_frame = cv2.imread(target_path)
result = process_frame(source_face, target_frame)
cv2.imwrite(output_path, result)
else:
if modules.globals.many_faces:
update_status(
"Many faces enabled. Using first source image. Progressing...", NAME
)
target_frame = cv2.imread(output_path)
result = process_frame_v2(target_frame)
cv2.imwrite(output_path, result)
source_face = get_one_face(cv2.imread(source_path))
target_frame = cv2.imread(target_path)
result = process_frame(source_face, target_frame)
cv2.imwrite(output_path, result)
def process_video(source_path: str, temp_frame_paths: List[str]) -> None:
if modules.globals.map_faces and modules.globals.many_faces:
update_status(
"Many faces enabled. Using first source image. Progressing...", NAME
)
modules.processors.frame.core.process_video(
source_path, temp_frame_paths, process_frames
)
def create_lower_mouth_mask(
face: Face, frame: Frame
) -> (np.ndarray, np.ndarray, tuple, np.ndarray):
mask = np.zeros(frame.shape[:2], dtype=np.uint8)
mouth_cutout = None
landmarks = face.landmark_2d_106
if landmarks is not None:
# 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
lower_lip_order = [
65,
66,
62,
70,
69,
18,
19,
20,
21,
22,
23,
24,
0,
8,
7,
6,
5,
4,
3,
2,
65,
]
lower_lip_landmarks = landmarks[lower_lip_order].astype(
np.float32
) # Use float for precise calculations
# Calculate the center of the landmarks
center = np.mean(lower_lip_landmarks, axis=0)
# Expand the landmarks outward
expansion_factor = (
1 + modules.globals.mask_down_size
) # Adjust this for more or less expansion
expanded_landmarks = (lower_lip_landmarks - center) * expansion_factor + center
# Extend the top lip part
toplip_indices = [
20,
0,
1,
2,
3,
4,
5,
] # Indices for landmarks 2, 65, 66, 62, 70, 69, 18
toplip_extension = (
modules.globals.mask_size * 0.5
) # Adjust this factor to control the extension
for idx in toplip_indices:
direction = expanded_landmarks[idx] - center
direction = direction / np.linalg.norm(direction)
expanded_landmarks[idx] += direction * toplip_extension
# Extend the bottom part (chin area)
chin_indices = [
11,
12,
13,
14,
15,
16,
] # Indices for landmarks 21, 22, 23, 24, 0, 8
chin_extension = 2 * 0.2 # Adjust this factor to control the extension
for idx in chin_indices:
expanded_landmarks[idx][1] += (
expanded_landmarks[idx][1] - center[1]
) * chin_extension
# Convert back to integer coordinates
expanded_landmarks = expanded_landmarks.astype(np.int32)
# Calculate bounding box for the expanded lower mouth
min_x, min_y = np.min(expanded_landmarks, axis=0)
max_x, max_y = np.max(expanded_landmarks, axis=0)
# Add some padding to the bounding box
padding = int((max_x - min_x) * 0.1) # 10% padding
min_x = max(0, min_x - padding)
min_y = max(0, min_y - padding)
max_x = min(frame.shape[1], max_x + padding)
max_y = min(frame.shape[0], max_y + padding)
# Ensure the bounding box dimensions are valid
if max_x <= min_x or max_y <= min_y:
if (max_x - min_x) <= 1:
max_x = min_x + 1
if (max_y - min_y) <= 1:
max_y = min_y + 1
# Create the mask
mask_roi = np.zeros((max_y - min_y, max_x - min_x), dtype=np.uint8)
cv2.fillPoly(mask_roi, [expanded_landmarks - [min_x, min_y]], 255)
# Apply Gaussian blur to soften the mask edges
mask_roi = cv2.GaussianBlur(mask_roi, (15, 15), 5)
# Place the mask ROI in the full-sized mask
mask[min_y:max_y, min_x:max_x] = mask_roi
# Extract the masked area from the frame
mouth_cutout = frame[min_y:max_y, min_x:max_x].copy()
# Return the expanded lower lip polygon in original frame coordinates
lower_lip_polygon = expanded_landmarks
return mask, mouth_cutout, (min_x, min_y, max_x, max_y), lower_lip_polygon
def draw_mouth_mask_visualization(
frame: Frame, face: Face, mouth_mask_data: tuple
) -> Frame:
landmarks = face.landmark_2d_106
if landmarks is not None and mouth_mask_data is not None:
mask, mouth_cutout, (min_x, min_y, max_x, max_y), lower_lip_polygon = (
mouth_mask_data
)
vis_frame = frame.copy()
# Ensure coordinates are within frame bounds
height, width = vis_frame.shape[:2]
min_x, min_y = max(0, min_x), max(0, min_y)
max_x, max_y = min(width, max_x), min(height, max_y)
# Adjust mask to match the region size
mask_region = mask[0 : max_y - min_y, 0 : max_x - min_x]
# Remove the color mask overlay
# color_mask = cv2.applyColorMap((mask_region * 255).astype(np.uint8), cv2.COLORMAP_JET)
# Ensure shapes match before blending
vis_region = vis_frame[min_y:max_y, min_x:max_x]
# Remove blending with color_mask
# if vis_region.shape[:2] == color_mask.shape[:2]:
# blended = cv2.addWeighted(vis_region, 0.7, color_mask, 0.3, 0)
# vis_frame[min_y:max_y, min_x:max_x] = blended
# Draw the lower lip polygon
cv2.polylines(vis_frame, [lower_lip_polygon], True, (0, 255, 0), 2)
# Remove the red box
# cv2.rectangle(vis_frame, (min_x, min_y), (max_x, max_y), (0, 0, 255), 2)
# Visualize the feathered mask
feather_amount = max(
1,
min(
30,
(max_x - min_x) // modules.globals.mask_feather_ratio,
(max_y - min_y) // modules.globals.mask_feather_ratio,
),
)
# Ensure kernel size is odd
kernel_size = 2 * feather_amount + 1
feathered_mask = cv2.GaussianBlur(
mask_region.astype(float), (kernel_size, kernel_size), 0
)
feathered_mask = (feathered_mask / feathered_mask.max() * 255).astype(np.uint8)
# Remove the feathered mask color overlay
# color_feathered_mask = cv2.applyColorMap(feathered_mask, cv2.COLORMAP_VIRIDIS)
# Ensure shapes match before blending feathered mask
# if vis_region.shape == color_feathered_mask.shape:
# blended_feathered = cv2.addWeighted(vis_region, 0.7, color_feathered_mask, 0.3, 0)
# vis_frame[min_y:max_y, min_x:max_x] = blended_feathered
# Add labels
cv2.putText(
vis_frame,
"Lower Mouth Mask",
(min_x, min_y - 10),
cv2.FONT_HERSHEY_SIMPLEX,
0.5,
(255, 255, 255),
1,
)
cv2.putText(
vis_frame,
"Feathered Mask",
(min_x, max_y + 20),
cv2.FONT_HERSHEY_SIMPLEX,
0.5,
(255, 255, 255),
1,
)
return vis_frame
return frame
def apply_mouth_area(
frame: np.ndarray,
mouth_cutout: np.ndarray,
mouth_box: tuple,
face_mask: np.ndarray,
mouth_polygon: np.ndarray,
) -> np.ndarray:
min_x, min_y, max_x, max_y = mouth_box
box_width = max_x - min_x
box_height = max_y - min_y
if (
mouth_cutout is None
or box_width is None
or box_height is None
or face_mask is None
or mouth_polygon is None
):
return frame
try:
resized_mouth_cutout = cv2.resize(mouth_cutout, (box_width, box_height))
roi = frame[min_y:max_y, min_x:max_x]
if roi.shape != resized_mouth_cutout.shape:
resized_mouth_cutout = cv2.resize(
resized_mouth_cutout, (roi.shape[1], roi.shape[0])
)
color_corrected_mouth = apply_color_transfer(resized_mouth_cutout, roi)
# Use the provided mouth polygon to create the mask
polygon_mask = np.zeros(roi.shape[:2], dtype=np.uint8)
adjusted_polygon = mouth_polygon - [min_x, min_y]
cv2.fillPoly(polygon_mask, [adjusted_polygon], 255)
# Apply feathering to the polygon mask
feather_amount = min(
30,
box_width // modules.globals.mask_feather_ratio,
box_height // modules.globals.mask_feather_ratio,
)
feathered_mask = cv2.GaussianBlur(
polygon_mask.astype(float), (0, 0), feather_amount
)
feathered_mask = feathered_mask / feathered_mask.max()
face_mask_roi = face_mask[min_y:max_y, min_x:max_x]
combined_mask = feathered_mask * (face_mask_roi / 255.0)
combined_mask = combined_mask[:, :, np.newaxis]
blended = (
color_corrected_mouth * combined_mask + roi * (1 - combined_mask)
).astype(np.uint8)
# Apply face mask to blended result
face_mask_3channel = (
np.repeat(face_mask_roi[:, :, np.newaxis], 3, axis=2) / 255.0
)
final_blend = blended * face_mask_3channel + roi * (1 - face_mask_3channel)
frame[min_y:max_y, min_x:max_x] = final_blend.astype(np.uint8)
except Exception as e:
pass
return frame
def create_face_mask(face: Face, frame: Frame) -> np.ndarray:
mask = np.zeros(frame.shape[:2], dtype=np.uint8)
landmarks = face.landmark_2d_106
if landmarks is not None:
# Convert landmarks to int32
landmarks = landmarks.astype(np.int32)
# Extract facial features
right_side_face = landmarks[0:16]
left_side_face = landmarks[17:32]
right_eye = landmarks[33:42]
right_eye_brow = landmarks[43:51]
left_eye = landmarks[87:96]
left_eye_brow = landmarks[97:105]
# Calculate forehead extension
right_eyebrow_top = np.min(right_eye_brow[:, 1])
left_eyebrow_top = np.min(left_eye_brow[:, 1])
eyebrow_top = min(right_eyebrow_top, left_eyebrow_top)
face_top = np.min([right_side_face[0, 1], left_side_face[-1, 1]])
forehead_height = face_top - eyebrow_top
extended_forehead_height = int(forehead_height * 5.0) # Extend by 50%
# Create forehead points
forehead_left = right_side_face[0].copy()
forehead_right = left_side_face[-1].copy()
forehead_left[1] -= extended_forehead_height
forehead_right[1] -= extended_forehead_height
# Combine all points to create the face outline
face_outline = np.vstack(
[
[forehead_left],
right_side_face,
left_side_face[
::-1
], # Reverse left side to create a continuous outline
[forehead_right],
]
)
# Calculate padding
padding = int(
np.linalg.norm(right_side_face[0] - left_side_face[-1]) * 0.05
) # 5% of face width
# Create a slightly larger convex hull for padding
hull = cv2.convexHull(face_outline)
hull_padded = []
for point in hull:
x, y = point[0]
center = np.mean(face_outline, axis=0)
direction = np.array([x, y]) - center
direction = direction / np.linalg.norm(direction)
padded_point = np.array([x, y]) + direction * padding
hull_padded.append(padded_point)
hull_padded = np.array(hull_padded, dtype=np.int32)
# Fill the padded convex hull
cv2.fillConvexPoly(mask, hull_padded, 255)
# Smooth the mask edges
mask = cv2.GaussianBlur(mask, (5, 5), 3)
return mask
def apply_color_transfer(source, target):
"""
Apply color transfer from target to source image
"""
source = cv2.cvtColor(source, cv2.COLOR_BGR2LAB).astype("float32")
target = cv2.cvtColor(target, cv2.COLOR_BGR2LAB).astype("float32")
source_mean, source_std = cv2.meanStdDev(source)
target_mean, target_std = cv2.meanStdDev(target)
# Reshape mean and std to be broadcastable
source_mean = source_mean.reshape(1, 1, 3)
source_std = source_std.reshape(1, 1, 3)
target_mean = target_mean.reshape(1, 1, 3)
target_std = target_std.reshape(1, 1, 3)
# Perform the color transfer
source = (source - source_mean) * (target_std / source_std) + target_mean
return cv2.cvtColor(np.clip(source, 0, 255).astype("uint8"), cv2.COLOR_LAB2BGR)
modules.processors.frame.core.process_video(source_path, temp_frame_paths, process_frames)
@@ -0,0 +1,197 @@
import threading
import traceback
from typing import Any, List
import cv2
import os
import modules.globals
import modules.processors.frame.core
from modules.core import update_status
from modules.face_analyser import get_one_face
from modules.utilities import conditional_download, resolve_relative_path, is_image, is_video
import numpy as np
NAME = 'DLC.SUPER-RESOLUTION'
THREAD_SEMAPHORE = threading.Semaphore()
# Singleton class for Super-Resolution
class SuperResolutionModel:
_instance = None
_lock = threading.Lock()
def __init__(self, sr_model_path: str = f'ESPCN_x{modules.globals.sr_scale_factor}.pb'):
if SuperResolutionModel._instance is not None:
raise Exception("This class is a singleton!")
self.sr = cv2.dnn_superres.DnnSuperResImpl_create()
self.model_path = os.path.join(resolve_relative_path('../models'), sr_model_path)
if not os.path.exists(self.model_path):
raise FileNotFoundError(f"Super-resolution model not found at {self.model_path}")
try:
self.sr.readModel(self.model_path)
self.sr.setModel("espcn", modules.globals.sr_scale_factor) # Using ESPCN with 2,3 or 4x upscaling
except Exception as e:
print(f"Error during super-resolution model initialization: {e}")
raise e
@classmethod
def get_instance(cls, sr_model_path: str = f'ESPCN_x{modules.globals.sr_scale_factor}.pb'):
if cls._instance is None:
with cls._lock:
if cls._instance is None:
try:
cls._instance = cls(sr_model_path)
except Exception as e:
raise RuntimeError(f"Failed to initialize SuperResolution: {str(e)}")
return cls._instance
def pre_check() -> bool:
"""
Checks and downloads necessary models before starting the face swapper.
"""
download_directory_path = resolve_relative_path('../models')
# Download the super-resolution model as well
conditional_download(download_directory_path, [
f'https://huggingface.co/spaces/PabloGabrielSch/AI_Resolution_Upscaler_And_Resizer/resolve/bcd13b766a9499196e8becbe453c4a848673b3b6/models/ESPCN_x{modules.globals.sr_scale_factor}.pb'
])
return True
def pre_start() -> bool:
if not is_image(modules.globals.source_path):
update_status('Select an image for source path.', NAME)
return False
elif not get_one_face(cv2.imread(modules.globals.source_path)):
update_status('No face detected in the source path.', NAME)
return False
if not is_image(modules.globals.target_path) and not is_video(modules.globals.target_path):
update_status('Select an image or video for target path.', NAME)
return False
return True
def apply_super_resolution(image: np.ndarray) -> np.ndarray:
"""
Applies super-resolution to the given image using the provided super-resolver.
Args:
image (np.ndarray): The input image to enhance.
sr_model_path (str): ESPCN model path for super-resolution.
Returns:
np.ndarray: The super-resolved image.
"""
with THREAD_SEMAPHORE:
sr_model = SuperResolutionModel.get_instance()
if sr_model is None:
print("Super-resolution model is not initialized.")
return image
try:
upscaled_image = sr_model.sr.upsample(image)
return upscaled_image
except Exception as e:
print(f"Error during super-resolution: {e}")
return image
def process_frame(frame: np.ndarray) -> np.ndarray:
"""
Processes a single frame by swapping the source face into detected target faces.
Args:
frame (np.ndarray): The target frame image.
Returns:
np.ndarray: The processed frame with swapped faces.
"""
# Apply super-resolution to the entire frame
frame = apply_super_resolution(frame)
return frame
def process_frames(source_path: str, temp_frame_paths: List[str], progress: Any = None) -> None:
"""
Processes multiple frames by swapping the source face into each target frame.
Args:
source_path (str): Path to the source image.
temp_frame_paths (List[str]): List of paths to target frame images.
progress (Any, optional): Progress tracker. Defaults to None.
"""
for idx, temp_frame_path in enumerate(temp_frame_paths):
frame = cv2.imread(temp_frame_path)
if frame is None:
print(f"Failed to load frame from {temp_frame_path}")
continue
try:
result = process_frame(frame)
cv2.imwrite(temp_frame_path, result)
except Exception as exception:
traceback.print_exc()
print(f"Error processing frame {temp_frame_path}: {exception}")
if progress:
progress.update(1)
def upscale_image(image: np.ndarray, scaling_factor: int = 2) -> np.ndarray:
"""
Upscales the given image by the specified scaling factor.
Args:
image (np.ndarray): The input image to upscale.
scaling_factor (int): The factor by which to upscale the image.
Returns:
np.ndarray: The upscaled image.
"""
height, width = image.shape[:2]
new_size = (width * scaling_factor, height * scaling_factor)
upscaled_image = cv2.resize(image, new_size, interpolation=cv2.INTER_CUBIC)
return upscaled_image
def process_image(source_path: str, target_path: str, output_path: str) -> None:
"""
Processes a single image by swapping the source face into the target image.
Args:
source_path (str): Path to the source image.
target_path (str): Path to the target image.
output_path (str): Path to save the output image.
"""
source_image = cv2.imread(source_path)
if source_image is None:
print(f"Failed to load source image from {source_path}")
return
# Upscale the source image for better quality before face detection
source_image_upscaled = upscale_image(source_image, scaling_factor=2)
# Detect source face from the upscaled image
source_face = get_one_face(source_image_upscaled)
if source_face is None:
print("No source face detected.")
return
target_frame = cv2.imread(target_path)
if target_frame is None:
print(f"Failed to load target image from {target_path}")
return
# Process the frame
result = process_frame(target_frame)
# Save the processed frame
cv2.imwrite(output_path, result)
def process_video(source_path: str, temp_frame_paths: List[str]) -> None:
"""
Processes a video by swapping the source face into each frame.
Args:
source_path (str): Path to the source image.
temp_frame_paths (List[str]): List of paths to video frame images.
"""
modules.processors.frame.core.process_video(None, temp_frame_paths, process_frames)
+28 -95
View File
@@ -1,76 +1,57 @@
{
"CTk": {
"fg_color": ["gray95", "gray10"]
"fg_color": ["#FFFFFF", "#2D2D2D"]
},
"CTkToplevel": {
"fg_color": ["gray95", "gray10"]
"fg_color": ["#FFFFFF", "#2D2D2D"]
},
"CTkFrame": {
"corner_radius": 0,
"border_width": 0,
"fg_color": ["gray90", "gray13"],
"top_fg_color": ["gray85", "gray16"],
"border_color": ["gray65", "gray28"]
"fg_color": ["#F0F0F0", "#3C3C3C"],
"top_fg_color": ["#E0E0E0", "#4B4B4B"],
"border_color": ["#B0B0B0", "#5A5A5A"]
},
"CTkButton": {
"corner_radius": 0,
"border_width": 0,
"fg_color": ["#2aa666", "#1f538d"],
"hover_color": ["#3cb666", "#14375e"],
"border_color": ["#3e4a40", "#949A9F"],
"text_color": ["#f3faf6", "#f3faf6"],
"fg_color": ["#007ACC", "#007ACC"],
"hover_color": ["#005EA3", "#005EA3"],
"border_color": ["#004C8A", "#004C8A"],
"text_color": ["#FFFFFF", "#FFFFFF"],
"text_color_disabled": ["gray74", "gray60"]
},
"CTkLabel": {
"corner_radius": 0,
"fg_color": "transparent",
"text_color": ["gray14", "gray84"]
"text_color": ["#000000", "#FFFFFF"]
},
"CTkEntry": {
"corner_radius": 0,
"border_width": 2,
"fg_color": ["#F9F9FA", "#343638"],
"border_color": ["#979DA2", "#565B5E"],
"text_color": ["gray14", "gray84"],
"fg_color": ["#FFFFFF", "#333333"],
"border_color": ["#A0A0A0", "#5A5A5A"],
"text_color": ["#000000", "#FFFFFF"],
"placeholder_text_color": ["gray52", "gray62"]
},
"CTkCheckbox": {
"corner_radius": 0,
"border_width": 3,
"fg_color": ["#2aa666", "#1f538d"],
"border_color": ["#3e4a40", "#949A9F"],
"hover_color": ["#3cb666", "#14375e"],
"checkmark_color": ["#f3faf6", "gray90"],
"text_color": ["gray14", "gray84"],
"text_color_disabled": ["gray60", "gray45"]
},
"CTkSwitch": {
"corner_radius": 1000,
"border_width": 3,
"button_length": 0,
"fg_color": ["#939BA2", "#4A4D50"],
"progress_color": ["#2aa666", "#1f538d"],
"button_color": ["gray36", "#D5D9DE"],
"button_hover_color": ["gray20", "gray100"],
"text_color": ["gray14", "gray84"],
"button_color": ["#444444", "#D5D9DE"],
"button_hover_color": ["#333333", "#FFFFFF"],
"text_color": ["#000000", "#FFFFFF"],
"text_color_disabled": ["gray60", "gray45"]
},
"CTkRadiobutton": {
"corner_radius": 1000,
"border_width_checked": 6,
"border_width_unchecked": 3,
"CTkOptionMenu": {
"corner_radius": 0,
"fg_color": ["#2aa666", "#1f538d"],
"border_color": ["#3e4a40", "#949A9F"],
"hover_color": ["#3cb666", "#14375e"],
"text_color": ["gray14", "gray84"],
"text_color_disabled": ["gray60", "gray45"]
},
"CTkProgressBar": {
"corner_radius": 1000,
"border_width": 0,
"fg_color": ["#939BA2", "#4A4D50"],
"progress_color": ["#2aa666", "#1f538d"],
"border_color": ["gray", "gray"]
"button_color": ["#3cb666", "#14375e"],
"button_hover_color": ["#234567", "#1e2c40"],
"text_color": ["#FFFFFF", "#FFFFFF"],
"text_color_disabled": ["gray74", "gray60"]
},
"CTkSlider": {
"corner_radius": 1000,
@@ -82,59 +63,6 @@
"button_color": ["#2aa666", "#1f538d"],
"button_hover_color": ["#3cb666", "#14375e"]
},
"CTkOptionMenu": {
"corner_radius": 0,
"fg_color": ["#2aa666", "#1f538d"],
"button_color": ["#3cb666", "#14375e"],
"button_hover_color": ["#234567", "#1e2c40"],
"text_color": ["#f3faf6", "#f3faf6"],
"text_color_disabled": ["gray74", "gray60"]
},
"CTkComboBox": {
"corner_radius": 0,
"border_width": 2,
"fg_color": ["#F9F9FA", "#343638"],
"border_color": ["#979DA2", "#565B5E"],
"button_color": ["#979DA2", "#565B5E"],
"button_hover_color": ["#6E7174", "#7A848D"],
"text_color": ["gray14", "gray84"],
"text_color_disabled": ["gray50", "gray45"]
},
"CTkScrollbar": {
"corner_radius": 1000,
"border_spacing": 4,
"fg_color": "transparent",
"button_color": ["gray55", "gray41"],
"button_hover_color": ["gray40", "gray53"]
},
"CTkSegmentedButton": {
"corner_radius": 0,
"border_width": 2,
"fg_color": ["#979DA2", "gray29"],
"selected_color": ["#2aa666", "#1f538d"],
"selected_hover_color": ["#3cb666", "#14375e"],
"unselected_color": ["#979DA2", "gray29"],
"unselected_hover_color": ["gray70", "gray41"],
"text_color": ["#f3faf6", "#f3faf6"],
"text_color_disabled": ["gray74", "gray60"]
},
"CTkTextbox": {
"corner_radius": 0,
"border_width": 0,
"fg_color": ["gray100", "gray20"],
"border_color": ["#979DA2", "#565B5E"],
"text_color": ["gray14", "gray84"],
"scrollbar_button_color": ["gray55", "gray41"],
"scrollbar_button_hover_color": ["gray40", "gray53"]
},
"CTkScrollableFrame": {
"label_fg_color": ["gray80", "gray21"]
},
"DropdownMenu": {
"fg_color": ["gray90", "gray20"],
"hover_color": ["gray75", "gray28"],
"text_color": ["gray14", "gray84"]
},
"CTkFont": {
"macOS": {
"family": "Avenir",
@@ -152,7 +80,12 @@
"weight": "normal"
}
},
"DropdownMenu": {
"fg_color": ["#FFFFFF", "#2D2D2D"],
"hover_color": ["#E0E0E0", "#4B4B4B"],
"text_color": ["#000000", "#FFFFFF"]
},
"URL": {
"text_color": ["gray74", "gray60"]
"text_color": ["#007ACC", "#1E90FF"]
}
}
+272 -984
View File
File diff suppressed because it is too large Load Diff
+54 -126
View File
@@ -5,205 +5,133 @@ import platform
import shutil
import ssl
import subprocess
import urllib
import urllib.request
from pathlib import Path
from typing import List, Any
from tqdm import tqdm
import modules.globals
TEMP_FILE = "temp.mp4"
TEMP_DIRECTORY = "temp"
TEMP_FILE = 'temp.mp4'
TEMP_DIRECTORY = 'temp'
# monkey patch ssl for mac
if platform.system().lower() == "darwin":
# Monkey patch SSL for macOS to handle issues with some HTTPS requests
if platform.system().lower() == 'darwin':
ssl._create_default_https_context = ssl._create_unverified_context
def run_ffmpeg(args: List[str]) -> bool:
commands = [
"ffmpeg",
"-hide_banner",
"-hwaccel",
"auto",
"-loglevel",
modules.globals.log_level,
]
commands = ['ffmpeg', '-hide_banner', '-hwaccel', 'auto', '-loglevel', modules.globals.log_level]
commands.extend(args)
try:
subprocess.check_output(commands, stderr=subprocess.STDOUT)
return True
except Exception:
pass
except subprocess.CalledProcessError as e:
print(f"FFmpeg error: {e.output.decode()}")
return False
def detect_fps(target_path: str) -> float:
command = [
"ffprobe",
"-v",
"error",
"-select_streams",
"v:0",
"-show_entries",
"stream=r_frame_rate",
"-of",
"default=noprint_wrappers=1:nokey=1",
target_path,
'ffprobe', '-v', 'error', '-select_streams', 'v:0',
'-show_entries', 'stream=r_frame_rate',
'-of', 'default=noprint_wrappers=1:nokey=1', target_path
]
output = subprocess.check_output(command).decode().strip().split("/")
try:
output = subprocess.check_output(command).decode().strip().split('/')
numerator, denominator = map(int, output)
return numerator / denominator
except Exception:
pass
except (subprocess.CalledProcessError, ValueError):
print("Failed to detect FPS, defaulting to 30.0 FPS.")
return 30.0
def extract_frames(target_path: str) -> None:
temp_directory_path = get_temp_directory_path(target_path)
run_ffmpeg(
[
"-i",
target_path,
"-pix_fmt",
"rgb24",
os.path.join(temp_directory_path, "%04d.png"),
]
)
create_temp(target_path)
run_ffmpeg(['-i', target_path, '-pix_fmt', 'rgb24', os.path.join(temp_directory_path, '%04d.png')])
def create_video(target_path: str, fps: float = 30.0) -> None:
temp_output_path = get_temp_output_path(target_path)
temp_directory_path = get_temp_directory_path(target_path)
run_ffmpeg(
[
"-r",
str(fps),
"-i",
os.path.join(temp_directory_path, "%04d.png"),
"-c:v",
modules.globals.video_encoder,
"-crf",
str(modules.globals.video_quality),
"-pix_fmt",
"yuv420p",
"-vf",
"colorspace=bt709:iall=bt601-6-625:fast=1",
"-y",
temp_output_path,
]
)
run_ffmpeg([
'-r', str(fps), '-i', os.path.join(temp_directory_path, '%04d.png'),
'-c:v', modules.globals.video_encoder,
'-crf', str(modules.globals.video_quality),
'-pix_fmt', 'yuv420p',
'-vf', 'colorspace=bt709:iall=bt601-6-625:fast=1',
'-y', temp_output_path
])
def restore_audio(target_path: str, output_path: str) -> None:
temp_output_path = get_temp_output_path(target_path)
done = run_ffmpeg(
[
"-i",
temp_output_path,
"-i",
target_path,
"-c:v",
"copy",
"-map",
"0:v:0",
"-map",
"1:a:0",
"-y",
output_path,
]
)
done = run_ffmpeg([
'-i', temp_output_path, '-i', target_path,
'-c:v', 'copy', '-map', '0:v:0', '-map', '1:a:0', '-y', output_path
])
if not done:
move_temp(target_path, output_path)
def get_temp_frame_paths(target_path: str) -> List[str]:
temp_directory_path = get_temp_directory_path(target_path)
return glob.glob((os.path.join(glob.escape(temp_directory_path), "*.png")))
return glob.glob(os.path.join(glob.escape(temp_directory_path), '*.png'))
def get_temp_directory_path(target_path: str) -> str:
target_name, _ = os.path.splitext(os.path.basename(target_path))
target_directory_path = os.path.dirname(target_path)
return os.path.join(target_directory_path, TEMP_DIRECTORY, target_name)
target_name = Path(target_path).stem
target_directory_path = Path(target_path).parent
return str(target_directory_path / TEMP_DIRECTORY / target_name)
def get_temp_output_path(target_path: str) -> str:
temp_directory_path = get_temp_directory_path(target_path)
return os.path.join(temp_directory_path, TEMP_FILE)
return str(Path(temp_directory_path) / TEMP_FILE)
def normalize_output_path(source_path: str, target_path: str, output_path: str) -> Any:
if source_path and target_path:
source_name, _ = os.path.splitext(os.path.basename(source_path))
target_name, target_extension = os.path.splitext(os.path.basename(target_path))
if os.path.isdir(output_path):
return os.path.join(
output_path, source_name + "-" + target_name + target_extension
)
def normalize_output_path(source_path: str, target_path: str, output_path: str) -> str:
if source_path and target_path and os.path.isdir(output_path):
source_name = Path(source_path).stem
target_name = Path(target_path).stem
target_extension = Path(target_path).suffix
return str(Path(output_path) / f"{source_name}-{target_name}{target_extension}")
return output_path
def create_temp(target_path: str) -> None:
temp_directory_path = get_temp_directory_path(target_path)
Path(temp_directory_path).mkdir(parents=True, exist_ok=True)
def move_temp(target_path: str, output_path: str) -> None:
temp_output_path = get_temp_output_path(target_path)
if os.path.isfile(temp_output_path):
if os.path.isfile(output_path):
os.remove(output_path)
shutil.move(temp_output_path, output_path)
def clean_temp(target_path: str) -> None:
temp_directory_path = get_temp_directory_path(target_path)
parent_directory_path = os.path.dirname(temp_directory_path)
parent_directory_path = Path(temp_directory_path).parent
if not modules.globals.keep_frames and os.path.isdir(temp_directory_path):
shutil.rmtree(temp_directory_path)
if os.path.exists(parent_directory_path) and not os.listdir(parent_directory_path):
os.rmdir(parent_directory_path)
if parent_directory_path.exists() and not list(parent_directory_path.iterdir()):
parent_directory_path.rmdir()
def has_image_extension(image_path: str) -> bool:
return image_path.lower().endswith(("png", "jpg", "jpeg"))
return image_path.lower().endswith(('png', 'jpg', 'jpeg'))
def is_image(image_path: str) -> bool:
if image_path and os.path.isfile(image_path):
mimetype, _ = mimetypes.guess_type(image_path)
return bool(mimetype and mimetype.startswith("image/"))
return mimetype and mimetype.startswith('image/')
return False
def is_video(video_path: str) -> bool:
if video_path and os.path.isfile(video_path):
mimetype, _ = mimetypes.guess_type(video_path)
return bool(mimetype and mimetype.startswith("video/"))
return mimetype and mimetype.startswith('video/')
return False
def conditional_download(download_directory_path: str, urls: List[str]) -> None:
if not os.path.exists(download_directory_path):
os.makedirs(download_directory_path)
download_directory = Path(download_directory_path)
download_directory.mkdir(parents=True, exist_ok=True)
for url in urls:
download_file_path = os.path.join(
download_directory_path, os.path.basename(url)
)
if not os.path.exists(download_file_path):
request = urllib.request.urlopen(url) # type: ignore[attr-defined]
total = int(request.headers.get("Content-Length", 0))
with tqdm(
total=total,
desc="Downloading",
unit="B",
unit_scale=True,
unit_divisor=1024,
) as progress:
urllib.request.urlretrieve(url, download_file_path, reporthook=lambda count, block_size, total_size: progress.update(block_size)) # type: ignore[attr-defined]
download_file_path = download_directory / Path(url).name
if not download_file_path.exists():
with urllib.request.urlopen(url) as request:
total = int(request.headers.get('Content-Length', 0))
with tqdm(total=total, desc='Downloading', unit='B', unit_scale=True, unit_divisor=1024) as progress:
urllib.request.urlretrieve(url, download_file_path, reporthook=lambda count, block_size, total_size: progress.update(block_size))
def resolve_relative_path(path: str) -> str:
return os.path.abspath(os.path.join(os.path.dirname(__file__), path))
return str(Path(__file__).parent / path)
-94
View File
@@ -1,94 +0,0 @@
import cv2
import numpy as np
from typing import Optional, Tuple, Callable
import platform
import threading
# Only import Windows-specific library if on Windows
if platform.system() == "Windows":
from pygrabber.dshow_graph import FilterGraph
class VideoCapturer:
def __init__(self, device_index: int):
self.device_index = device_index
self.frame_callback = None
self._current_frame = None
self._frame_ready = threading.Event()
self.is_running = False
self.cap = None
# Initialize Windows-specific components if on Windows
if platform.system() == "Windows":
self.graph = FilterGraph()
# Verify device exists
devices = self.graph.get_input_devices()
if self.device_index >= len(devices):
raise ValueError(
f"Invalid device index {device_index}. Available devices: {len(devices)}"
)
def start(self, width: int = 960, height: int = 540, fps: int = 60) -> bool:
"""Initialize and start video capture"""
try:
if platform.system() == "Windows":
# Windows-specific capture methods
capture_methods = [
(self.device_index, cv2.CAP_DSHOW), # Try DirectShow first
(self.device_index, cv2.CAP_ANY), # Then try default backend
(-1, cv2.CAP_ANY), # Try -1 as fallback
(0, cv2.CAP_ANY), # Finally try 0 without specific backend
]
for dev_id, backend in capture_methods:
try:
self.cap = cv2.VideoCapture(dev_id, backend)
if self.cap.isOpened():
break
self.cap.release()
except Exception:
continue
else:
# Unix-like systems (Linux/Mac) capture method
self.cap = cv2.VideoCapture(self.device_index)
if not self.cap or not self.cap.isOpened():
raise RuntimeError("Failed to open camera")
# Configure format
self.cap.set(cv2.CAP_PROP_FRAME_WIDTH, width)
self.cap.set(cv2.CAP_PROP_FRAME_HEIGHT, height)
self.cap.set(cv2.CAP_PROP_FPS, fps)
self.is_running = True
return True
except Exception as e:
print(f"Failed to start capture: {str(e)}")
if self.cap:
self.cap.release()
return False
def read(self) -> Tuple[bool, Optional[np.ndarray]]:
"""Read a frame from the camera"""
if not self.is_running or self.cap is None:
return False, None
ret, frame = self.cap.read()
if ret:
self._current_frame = frame
if self.frame_callback:
self.frame_callback(frame)
return True, frame
return False, None
def release(self) -> None:
"""Stop capture and release resources"""
if self.is_running and self.cap is not None:
self.cap.release()
self.is_running = False
self.cap = None
def set_frame_callback(self, callback: Callable[[np.ndarray], None]) -> None:
"""Set callback for frame processing"""
self.frame_callback = callback
+17 -15
View File
@@ -1,25 +1,27 @@
--extra-index-url https://download.pytorch.org/whl/nightly/cu128
--extra-index-url https://download.pytorch.org/whl/cu118
numpy>=1.23.5,<2
typing-extensions>=4.8.0
opencv-python==4.11.0.86
onnx==1.17.0
cv2_enumerate_cameras==1.1.18.3
numpy==1.23.5
opencv-contrib-python==4.10.0.84
onnx==1.16.0
insightface==0.7.3
psutil==5.9.8
tk==0.1.0
customtkinter==5.2.2
pillow==11.1.0
torch; sys_platform != 'darwin'
torch; sys_platform == 'darwin'
torchvision; sys_platform != 'darwin'
torchvision; sys_platform == 'darwin'
onnxruntime-silicon==1.21; sys_platform == 'darwin' and platform_machine == 'arm64'
onnxruntime-gpu==1.21; sys_platform != 'darwin'
tensorflow; sys_platform != 'darwin'
pillow==9.5.0
torch==2.0.1+cu118; sys_platform != 'darwin'
torch==2.0.1; sys_platform == 'darwin'
torchvision==0.15.2+cu118; sys_platform != 'darwin'
torchvision==0.15.2; sys_platform == 'darwin'
onnxruntime==1.18.0; sys_platform == 'darwin' and platform_machine != 'arm64'
onnxruntime-silicon==1.16.3; sys_platform == 'darwin' and platform_machine == 'arm64'
onnxruntime-gpu==1.18.0; sys_platform != 'darwin'
tensorflow==2.13.0rc1; sys_platform == 'darwin'
tensorflow==2.12.1; sys_platform != 'darwin'
opennsfw2==0.10.2
protobuf==4.23.2
tqdm==4.66.4
gfpgan==1.3.8
tkinterdnd2==0.4.2
pyobjc==9.1; sys_platform == 'darwin'
pygrabber==0.2
pyvirtualcam==0.12.0
pyobjc-framework-AVFoundation==10.3.1; sys_platform == 'darwin'
+1 -1
View File
@@ -1 +1 @@
python run.py --execution-provider cuda
python run.py --execution-provider cuda --execution-threads 60 --max-memory 60
-1
View File
@@ -1 +0,0 @@
python run.py --execution-provider dml
+1
View File
@@ -0,0 +1 @@
python run.py --execution-provider dml
+13
View File
@@ -0,0 +1,13 @@
@echo off
:: Installing Microsoft Visual C++ Runtime - all versions 1.0.1 if it's not already installed
choco install vcredist-all
:: Installing CUDA if it's not already installed
choco install cuda
:: Inatalling ffmpeg if it's not already installed
choco install ffmpeg
:: Installing Python if it's not already installed
choco install python -y
:: Assuming successful installation, we ensure pip is upgraded
python -m ensurepip --upgrade
:: Use pip to install the packages listed in 'requirements.txt'
pip install -r requirements.txt
+125
View File
@@ -0,0 +1,125 @@
@echo off
setlocal EnableDelayedExpansion
:: 1. Setup your platform
echo Setting up your platform...
call :check_installation python "Python 3.10 or later"
call :check_installation pip "Pip"
call :install_if_missing git "Git" "winget install --id Git.Git -e --source winget"
call :install_if_missing ffmpeg "FFMPEG" "winget install --id Gyan.FFmpeg -e --source winget"
:: Visual Studio 2022 Runtimes
echo Installing Visual Studio 2022 Runtimes...
winget install --id Microsoft.VC++2015-2022Redist-x64 -e --source winget
:: 2. Clone Repository
call :clone_repository "https://github.com/iVideoGameBoss/iRoopDeepFaceCam.git" "iRoopDeepFaceCam"
:: 3. Download Models
echo Downloading models...
if not exist models mkdir models
curl -L -o models\GFPGANv1.4.pth https://huggingface.co/ivideogameboss/iroopdeepfacecam/resolve/main/GFPGANv1.4.pth
curl -L -o models\inswapper_128_fp16.onnx https://huggingface.co/ivideogameboss/iroopdeepfacecam/resolve/main/inswapper_128_fp16.onnx
:: 4. Install dependencies
echo Creating a virtual environment...
python -m venv venv
call venv\Scripts\activate.bat
echo Installing required Python packages...
pip install --upgrade pip
pip install -r requirements.txt
echo Setup complete. You can now run the application.
:menu
:: GPU Acceleration Options
echo.
echo Choose the GPU Acceleration Option if applicable:
echo 1. CUDA (Nvidia)
echo 2. CoreML (Apple Silicon)
echo 3. CoreML (Apple Legacy)
echo 4. DirectML (Windows)
echo 5. OpenVINO (Intel)
echo 6. None
set /p choice="Enter your choice (1-6): "
set "exec_provider="
call :set_execution_provider %choice%
:end_choice
echo.
echo GPU Acceleration setup complete.
echo Selected provider: !exec_provider!
echo.
:: Run the application
if defined exec_provider (
echo Running the application with !exec_provider! execution provider...
python run.py --execution-provider !exec_provider!
) else (
echo Running the application...
python run.py
)
:: Deactivate the virtual environment
call venv\Scripts\deactivate.bat
echo.
echo Script execution completed.
pause
exit /b
:check_installation
where %1 >nul 2>&1
if %ERRORLEVEL% neq 0 (
echo %2 is not installed. Please install %2.
pause
exit /b
)
:install_if_missing
where %1 >nul 2>&1
if %ERRORLEVEL% neq 0 (
echo %2 is not installed. Installing %2...
%3
)
:clone_repository
if exist %2 (
echo %2 directory already exists.
set /p overwrite="Do you want to overwrite? (Y/N): "
if /i "%overwrite%"=="Y" (
rmdir /s /q %2
git clone %1
) else (
echo Skipping clone, using existing directory.
)
) else (
git clone %1
)
:set_execution_provider
if "%1"=="1" (
call :install_onnxruntime "onnxruntime-gpu" "1.16.3" "cuda"
) else if "%1"=="2" (
call :install_onnxruntime "onnxruntime-silicon" "1.13.1" "coreml"
) else if "%1"=="3" (
call :install_onnxruntime "onnxruntime-coreml" "1.13.1" "coreml"
) else if "%1"=="4" (
call :install_onnxruntime "onnxruntime-directml" "1.15.1" "directml"
) else if "%1"=="5" (
call :install_onnxruntime "onnxruntime-openvino" "1.15.0" "openvino"
) else if "%1"=="6" (
echo Skipping GPU acceleration setup.
set "exec_provider=none"
) else (
echo Invalid choice. Please try again.
goto menu
)
:install_onnxruntime
echo Installing %1 dependencies...
pip uninstall -y onnxruntime %1
pip install %1==%2
set "exec_provider=%3"
goto end_choice