Skip to content

Commit ec12665

Browse files
henryruhsphineas-ptaAntwaneBEamonn A. SweeneyMoeblack
authoredJun 20, 2023
Next (s0md3v#564)
* ffmpeg platform-agnostic hardware-acceleration * clear CUDA cache after swapping on low VRAM + ffmpeg cuda acceleration, clearing cache prevent cuda out-of-memory error * check torch gpu before clearing cache * torch check nvidia only * syntax error * Adjust comment * Normalize ARGS * Remove path normalization * Remove args overrides * Run test on Linux and Windows * Run test on Linux and Windows * Run test on Linux and Windows * Run test on Linux and Windows * Run test on Linux and Windows * Run test on Linux and Windows * Run test on Linux and Windows * Revert to Ubuntu test only as Windows hangs * Simplified the way to maintain aspect ratio of the preview, and maintaining aspect ratio of the miniatures * Change face and target images from contain to fit * Improve status output * Massive utilities and core refactoring * Fix sound * Fix sound part2 * Fix more * Move every UI related thing to ui.py * Refactor UI * Introduce render_video_preview() * Add preview back part1 * Add preview back part2, Introduce --video-quality for CLI * Get the preview working * Couple if minor UI fixes * Add video encoder via CLI * Change default video quality, Integrate recent directories for UI * Move temporary files to temp/{target-name} * Fix fps detection * Rename method * Introduce suggest methods for args defaults, output mode and core/threads count via postfix * Fix max_memory and output memory in progress bar too * Turns out mac has a different memory unit * Add typing to swapper * Fix FileNotFoundError while deleting temp * Updated requirements.txt for macs. (cherry picked from commit fd00a18) * Doing bunch of renaming and typing * Just a cosmetic * Doing bunch of renaming again * Introduce execution provider to support DirectML * enhancer update * remove useless code * remove useless code * remove useless code * fix * reslove some errors in code review. * methods rename * del Unused import * recover the conditional installation for darwin! * recover the import module * del try catch in unrelate codes * fix error in argument and potential infinity loop * remove the ROCM check before face-enhancer * fix lint error * add the process for image * conditional process according to --frame-processor * Hotfix usage of --frame-processor face-swapper face-enhancer * Auto download models * typo fixed * Fix framerate and audio sync issues * Limit the video encoders according to -crf support * Limit the video encoders according to -crf support part2 * Migrate to theme based UI using customtkinter * Show full preview according to video frames total * Simplify is_image and is_video, close preview on source/target change, show preview slider on video only, fix start button error * Fix linter * Use float over int for more accurate fps * introduce a pre_check() to enhancer... * update * Update utilities.py * move the model_path to the method * Fix model paths * Fix linter * Fix images scaling * Update onnxruntime-silicon and monkey patch ssl for mac * Downgrade onnxruntime-silicon again * Introduce a first version of CONTRIBUTING.md * Update CONTRIBUTING.md * Add libvpx-vp9 to video encoder's choices * Update CONTRIBUTING.md * Migrate gpu flags to execution flags * Fix linter * Encode and decode execution providers for easier usage * Fix comment * Update CLI usage * Fixed wrong frame colors for preview * Introduce dynamic frame procesors * Remove unused imports * Use different structure * modified: roop/core.py , enhancer and swapper * fix link error * modified core.py ui.py frame/core.py * fix get_frame_processors_modules() * fix lint error in core.py * fix face_enhancer.py * fix enhancer.py * fix ui.py * modified: roop/ui.py * fix ui.py * Update ui.py * Fix preview to work with multiple frame processors * Remove multi processing as the performance is equal but memory consumtion better * Extract common methods to processors.frame.core * Add comments and todos * Minor fixes * Minor fixes * Limit enhancer to 1 thread but restore afterwards * Limit enhancer to 1 thread but restore afterwards * Update README and GUI demo * Implementation check for frame processors * Improve path validation * Introduce normalize output path * Fix GUI startup * Introduce pre_start() and move globals check to pre_check() * Flip the hooks and update CLI usage * Introduce bool returns for pre_check() and pre_start(), Scope for terminal output * Cast deprecated args and warn the user * Migrate to ctk.CTkImage * readme update: old keys fixed * Unused import made CI test to fail * Give gfpgan a shot * Update dependencies * Update dependencies * Update dependencies * Update dependencies * Fix ci (s0md3v#498) * Use different dependencies for CI * Use different dependencies for CI * Use different dependencies for CI * Use different dependencies for CI * Use different dependencies for CI * Fix preview (s0md3v#499) * Minor changes * Fix override of files using restore_audio() * Host the models in our huggingface space * Remove everything source face related from enhancer (s0md3v#510) * Remove everything source face related from enhancer * Disable upscale for enhancer to double fps * Using futures for multi threading * Introduce predicter (s0md3v#530) * Hotfix predicter * Fix square brackets in the target path (s0md3v#532) * fixes the problem with square brackets in the target file path * fixes the problem with square brackets in the target file path * Ok, here is the fix without regexps * unused import * fix for ci * glob.escape fits here * Make multiple processors work with images * Fix output normalize for deprecated args * Make roop more or less type-safe (s0md3v#541) * Make roop more or less type-safe * Fix ci.yml * Fix urllib type error * Rename globals in ui * Update utilities.py (s0md3v#542) Updated the extraction process with the ffmpeg command that corrects the colorspace while extracting to png, and corrected the ffmpeg command, adding '-pix_fmt', 'rgb24', '-sws_flags', '+accurate_rnd+full_chroma_int', '-colorspace', '1', '-color_primaries', '1', '-color_trc', '1' '-pix_fmt rgb24', means treat the image as RGB (or RGBA) '-sws_flags +accurate_rnd+full_chroma_int', means use full color and chroma subsampling instead of 4:2:0 '-colorspace 1', '-color_primaries 1', '-color_trc 1' put the metadata color tags to the png * Use GFPGANv1.4 for enhancer * Fixing the colorspace issue when writing the mp4 from the extracted pngs (s0md3v#550) * Update utilities.py Updated the extraction process with the ffmpeg command that corrects the colorspace while extracting to png, and corrected the ffmpeg command, adding '-pix_fmt', 'rgb24', '-sws_flags', '+accurate_rnd+full_chroma_int', '-colorspace', '1', '-color_primaries', '1', '-color_trc', '1' '-pix_fmt rgb24', means treat the image as RGB (or RGBA) '-sws_flags +accurate_rnd+full_chroma_int', means use full color and chroma subsampling instead of 4:2:0 '-colorspace 1', '-color_primaries 1', '-color_trc 1' put the metadata color tags to the png * Fixing color conversion from temp png sequence to mp4 '-sws_flags', 'spline+accurate_rnd+full_chroma_int', ' use full color and chroma subsampling -vf', 'colorspace=bt709:iall=bt601-6-625:fast=1', keep the same rec709 colorspace '-color_range', '1', '-colorspace', '1', '-color_primaries', '1', '-color_trc', '1', put the metadata color tags to the mp4 * Revert "Fixing the colorspace issue when writing the mp4 from the extracted pngs (s0md3v#550)" This reverts commit cf5f27d. * Revert "Update utilities.py (s0md3v#542)" This reverts commit d57279c. * Restore colorspace restoring * Add metadata to cli and ui * Introduce Face and Frame typing * Update CLI usage --------- Co-authored-by: Phan Tuấn Anh <[email protected]> Co-authored-by: Antoine Buchser <[email protected]> Co-authored-by: Eamonn A. Sweeney <[email protected]> Co-authored-by: Moeblack <Moeblack@[email protected]> Co-authored-by: Pozitronik <[email protected]> Co-authored-by: Pikachu~~~ <[email protected]> Co-authored-by: K1llM@n <[email protected]> Co-authored-by: NickPittas <[email protected]>
1 parent 7010822 commit ec12665

29 files changed

+1079
-748
lines changed
 

‎.flake8

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
[flake8]
2-
select = E3, E4, F
2+
select = E3, E4, F
3+
per-file-ignores = roop/core.py:E402
File renamed without changes.

‎.github/workflows/ci.yml

+6-5
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,9 @@ jobs:
1313
with:
1414
python-version: 3.9
1515
- run: pip install flake8
16-
- run: flake8 run.py core
16+
- run: pip install mypy
17+
- run: flake8 run.py roop
18+
- run: mypy --config-file mypi.ini run.py roop
1719
test:
1820
runs-on: ubuntu-latest
1921
steps:
@@ -25,8 +27,7 @@ jobs:
2527
uses: actions/setup-python@v2
2628
with:
2729
python-version: 3.9
28-
- run: pip install -r requirements.txt gdown
29-
- run: gdown 13QpWFWJ37EB-nHrEOY64CEtQWY-tz7DZ
30-
- run: ./run.py -f=.github/examples/face.jpg -t=.github/examples/target.mp4 -o=.github/examples/output.mp4
31-
- run: ffmpeg -i .github/examples/snapshot.mp4 -i .github/examples/output.mp4 -filter_complex "psnr" -f null -
30+
- run: pip install -r requirements-ci.txt
31+
- run: python run.py -s=.github/examples/source.jpg -t=.github/examples/target.mp4 -o=.github/examples/output.mp4
32+
- run: ffmpeg -i .github/examples/snapshot.mp4 -i .github/examples/output.mp4 -filter_complex psnr -f null -
3233

‎.gitignore

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
11
.idea
2+
models
3+
temp
24
__pycache__
3-
*.onnx

‎CONTRIBUTING.md

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
## Pull Requests
2+
3+
### Do
4+
5+
- ...consider to fix bugs over adding features
6+
- ...one pull request for one feature or improvement
7+
- ...consult us about implementation details
8+
- ...proper testing before you submit your code
9+
- ...resolve failed CI pipelines
10+
11+
### Don't
12+
13+
- ...introduce fundamental changes in terms of software architecture
14+
- ...introduce OOP - we accept functional programming only
15+
- ...ignore given requirements or try to work around them
16+
- ...submit code to a development branch without consulting us
17+
- ...submit massive amount of code changes
18+
- ...submit a proof of concept
19+
- ...submit code that is using undocumented and private APIs
20+
- ...solve third party issues in our project
21+
- ...comment what your code does - use proper naming instead

‎README.md

+22-16
Original file line numberDiff line numberDiff line change
@@ -36,26 +36,32 @@ Additional command line arguments are given below:
3636
```
3737
options:
3838
-h, --help show this help message and exit
39-
-f SOURCE_IMG, --face SOURCE_IMG
40-
use this face
39+
-s SOURCE_PATH, --source SOURCE_PATH
40+
select an source image
4141
-t TARGET_PATH, --target TARGET_PATH
42-
replace this face
43-
-o OUTPUT_FILE, --output OUTPUT_FILE
44-
save output to this file
45-
--keep-fps maintain original fps
46-
--keep-frames keep frames directory
47-
--all-faces swap all faces in frame
42+
select an target image or video
43+
-o OUTPUT_PATH, --output OUTPUT_PATH
44+
select output file or directory
45+
--frame-processor {face_swapper,face_enhancer} [{face_swapper,face_enhancer} ...]
46+
pipeline of frame processors
47+
--keep-fps keep original fps
48+
--keep-audio keep original audio
49+
--keep-frames keep temporary frames
50+
--many-faces process every face
51+
--video-encoder {libx264,libx265,libvpx-vp9}
52+
adjust output video encoder
53+
--video-quality VIDEO_QUALITY
54+
adjust output video quality
4855
--max-memory MAX_MEMORY
49-
maximum amount of RAM in GB to be used
50-
--cpu-cores CPU_CORES
51-
number of CPU cores to use
52-
--gpu-threads GPU_THREADS
53-
number of threads to be use for the GPU
54-
--gpu-vendor {apple,amd,intel,nvidia}
55-
choice your GPU vendor
56+
maximum amount of RAM in GB
57+
--execution-provider {cpu,...} [{cpu,...} ...]
58+
execution provider
59+
--execution-threads EXECUTION_THREADS
60+
number of execution threads
61+
-v, --version show program's version number and exit
5662
```
5763

58-
Looking for a CLI mode? Using the -f/--face argument will make the program in cli mode.
64+
Looking for a CLI mode? Using the -s/--source argument will make the program in cli mode.
5965

6066
## Credits
6167
- [henryruhs](https://github.com/henryruhs): for being an irreplaceable contributor to the project

‎gui-demo.png

-11.7 KB
Loading

‎mypi.ini

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
[mypy]
2+
check_untyped_defs = True
3+
disallow_any_generics = True
4+
disallow_untyped_calls = True
5+
disallow_untyped_defs = True
6+
ignore_missing_imports = True
7+
strict_optional = False

‎requirements-ci.txt

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
numpy==1.23.5
2+
opencv-python==4.7.0.72
3+
onnx==1.14.0
4+
insightface==0.7.3
5+
psutil==5.9.5
6+
tk==0.1.0
7+
customtkinter==5.1.3
8+
torch==2.0.1
9+
torchvision==0.15.2
10+
onnxruntime==1.15.0
11+
tensorflow==2.12.0
12+
opennsfw2==0.10.2
13+
protobuf==4.23.2
14+
tqdm==4.65.0

‎requirements.txt

+5-1
Original file line numberDiff line numberDiff line change
@@ -6,14 +6,18 @@ onnx==1.14.0
66
insightface==0.7.3
77
psutil==5.9.5
88
tk==0.1.0
9+
customtkinter==5.1.3
910
pillow==9.5.0
1011
torch==2.0.1+cu118; sys_platform != 'darwin'
1112
torch==2.0.1; sys_platform == 'darwin'
13+
torchvision==0.15.2+cu118; sys_platform != 'darwin'
14+
torchvision==0.15.2; sys_platform == 'darwin'
1215
onnxruntime==1.15.0; sys_platform == 'darwin' and platform_machine != 'arm64'
1316
onnxruntime-silicon==1.13.1; sys_platform == 'darwin' and platform_machine == 'arm64'
1417
onnxruntime-gpu==1.15.0; sys_platform != 'darwin'
1518
tensorflow==2.13.0rc1; sys_platform == 'darwin'
1619
tensorflow==2.12.0; sys_platform != 'darwin'
1720
opennsfw2==0.10.2
1821
protobuf==4.23.2
19-
tqdm==4.65.0
22+
tqdm==4.65.0
23+
gfpgan==1.3.8

‎roop/__init__.py

-1
Original file line numberDiff line numberDiff line change
@@ -1 +0,0 @@
1-

‎roop/analyser.py

-27
This file was deleted.

‎roop/capturer.py

+20
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
from typing import Any
2+
import cv2
3+
4+
5+
def get_video_frame(video_path: str, frame_number: int = 0) -> Any:
6+
capture = cv2.VideoCapture(video_path)
7+
frame_total = capture.get(cv2.CAP_PROP_FRAME_COUNT)
8+
capture.set(cv2.CAP_PROP_POS_FRAMES, min(frame_total, frame_number - 1))
9+
has_frame, frame = capture.read()
10+
capture.release()
11+
if has_frame:
12+
return frame
13+
return None
14+
15+
16+
def get_video_frame_total(video_path: str) -> int:
17+
capture = cv2.VideoCapture(video_path)
18+
video_frame_total = int(capture.get(cv2.CAP_PROP_FRAME_COUNT))
19+
capture.release()
20+
return video_frame_total

‎roop/core.py

+200-242
Large diffs are not rendered by default.

‎roop/face_analyser.py

+31
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
from typing import Any
2+
import insightface
3+
4+
import roop.globals
5+
from roop.typing import Frame
6+
7+
FACE_ANALYSER = None
8+
9+
10+
def get_face_analyser() -> Any:
11+
global FACE_ANALYSER
12+
13+
if FACE_ANALYSER is None:
14+
FACE_ANALYSER = insightface.app.FaceAnalysis(name='buffalo_l', providers=roop.globals.execution_providers)
15+
FACE_ANALYSER.prepare(ctx_id=0, det_size=(640, 640))
16+
return FACE_ANALYSER
17+
18+
19+
def get_one_face(frame: Frame) -> Any:
20+
face = get_face_analyser().get(frame)
21+
try:
22+
return min(face, key=lambda x: x.bbox[0])
23+
except ValueError:
24+
return None
25+
26+
27+
def get_many_faces(frame: Frame) -> Any:
28+
try:
29+
return get_face_analyser().get(frame)
30+
except IndexError:
31+
return None

‎roop/globals.py

+15-9
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,17 @@
1-
import onnxruntime
1+
from typing import List
22

3-
all_faces = None
3+
source_path = None
4+
target_path = None
5+
output_path = None
6+
frame_processors: List[str] = []
7+
keep_fps = None
8+
keep_audio = None
9+
keep_frames = None
10+
many_faces = None
11+
video_encoder = None
12+
video_quality = None
13+
max_memory = None
14+
execution_providers: List[str] = []
15+
execution_threads = None
16+
headless = None
417
log_level = 'error'
5-
cpu_cores = None
6-
gpu_threads = None
7-
gpu_vendor = None
8-
providers = onnxruntime.get_available_providers()
9-
10-
if 'TensorrtExecutionProvider' in providers:
11-
providers.remove('TensorrtExecutionProvider')

‎roop/metadata.py

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
name = 'roop'
2+
version = '1.0.0'

‎roop/predicter.py

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
import numpy
2+
import opennsfw2
3+
from PIL import Image
4+
5+
from roop.typing import Frame
6+
7+
MAX_PROBABILITY = 0.85
8+
9+
10+
def predict_frame(target_frame: Frame) -> bool:
11+
image = Image.fromarray(target_frame)
12+
image = opennsfw2.preprocess_image(image, opennsfw2.Preprocessing.YAHOO)
13+
model = opennsfw2.make_open_nsfw_model()
14+
views = numpy.expand_dims(image, axis=0)
15+
_, probability = model.predict(views)[0]
16+
return probability > MAX_PROBABILITY
17+
18+
19+
def predict_image(target_path: str) -> bool:
20+
return opennsfw2.predict_image(target_path) > MAX_PROBABILITY
21+
22+
23+
def predict_video(target_path: str) -> bool:
24+
_, probabilities = opennsfw2.predict_video_frames(video_path=target_path, frame_interval=100)
25+
return any(probability > MAX_PROBABILITY for probability in probabilities)

‎roop/processors/__init__.py

Whitespace-only changes.

‎roop/processors/frame/__init__.py

Whitespace-only changes.

‎roop/processors/frame/core.py

+56
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
import sys
2+
import importlib
3+
from concurrent.futures import ThreadPoolExecutor
4+
from types import ModuleType
5+
from typing import Any, List, Callable
6+
from tqdm import tqdm
7+
8+
import roop
9+
10+
FRAME_PROCESSORS_MODULES: List[ModuleType] = []
11+
FRAME_PROCESSORS_INTERFACE = [
12+
'pre_check',
13+
'pre_start',
14+
'process_frame',
15+
'process_image',
16+
'process_video'
17+
]
18+
19+
20+
def load_frame_processor_module(frame_processor: str) -> Any:
21+
try:
22+
frame_processor_module = importlib.import_module(f'roop.processors.frame.{frame_processor}')
23+
for method_name in FRAME_PROCESSORS_INTERFACE:
24+
if not hasattr(frame_processor_module, method_name):
25+
sys.exit()
26+
except ImportError:
27+
sys.exit()
28+
return frame_processor_module
29+
30+
31+
def get_frame_processors_modules(frame_processors: List[str]) -> List[ModuleType]:
32+
global FRAME_PROCESSORS_MODULES
33+
34+
if not FRAME_PROCESSORS_MODULES:
35+
for frame_processor in frame_processors:
36+
frame_processor_module = load_frame_processor_module(frame_processor)
37+
FRAME_PROCESSORS_MODULES.append(frame_processor_module)
38+
return FRAME_PROCESSORS_MODULES
39+
40+
41+
def multi_process_frame(source_path: str, temp_frame_paths: List[str], process_frames: Callable[[str, List[str], Any], None], progress: Any = None) -> None:
42+
with ThreadPoolExecutor(max_workers=roop.globals.execution_threads) as executor:
43+
futures = []
44+
for path in temp_frame_paths:
45+
future = executor.submit(process_frames, source_path, [path], progress)
46+
futures.append(future)
47+
for future in futures:
48+
future.result()
49+
50+
51+
def process_video(source_path: str, frame_paths: list[str], process_frames: Callable[[str, List[str], Any], None]) -> None:
52+
progress_bar_format = '{l_bar}{bar}| {n_fmt}/{total_fmt} [{elapsed}<{remaining}, {rate_fmt}{postfix}]'
53+
total = len(frame_paths)
54+
with tqdm(total=total, desc='Processing', unit='frame', dynamic_ncols=True, bar_format=progress_bar_format) as progress:
55+
progress.set_postfix({'execution_providers': roop.globals.execution_providers, 'threads': roop.globals.execution_threads, 'memory': roop.globals.max_memory})
56+
multi_process_frame(source_path, frame_paths, process_frames, progress)
+75
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
from typing import Any, List
2+
import cv2
3+
import threading
4+
import gfpgan
5+
6+
import roop.globals
7+
import roop.processors.frame.core
8+
from roop.core import update_status
9+
from roop.face_analyser import get_one_face
10+
from roop.typing import Frame, Face
11+
from roop.utilities import conditional_download, resolve_relative_path, is_image, is_video
12+
13+
FACE_ENHANCER = None
14+
THREAD_SEMAPHORE = threading.Semaphore()
15+
THREAD_LOCK = threading.Lock()
16+
NAME = 'ROOP.FACE-ENHANCER'
17+
18+
19+
def pre_check() -> bool:
20+
download_directory_path = resolve_relative_path('../models')
21+
conditional_download(download_directory_path, ['https://huggingface.co/henryruhs/roop/resolve/main/GFPGANv1.4.pth'])
22+
return True
23+
24+
25+
def pre_start() -> bool:
26+
if not is_image(roop.globals.target_path) and not is_video(roop.globals.target_path):
27+
update_status('Select an image or video for target path.', NAME)
28+
return False
29+
return True
30+
31+
32+
def get_face_enhancer() -> Any:
33+
global FACE_ENHANCER
34+
35+
with THREAD_LOCK:
36+
if FACE_ENHANCER is None:
37+
model_path = resolve_relative_path('../models/GFPGANv1.4.pth')
38+
# todo: set models path https://github.com/TencentARC/GFPGAN/issues/399
39+
FACE_ENHANCER = gfpgan.GFPGANer(model_path=model_path, upscale=1) # type: ignore[attr-defined]
40+
return FACE_ENHANCER
41+
42+
43+
def enhance_face(temp_frame: Frame) -> Frame:
44+
with THREAD_SEMAPHORE:
45+
_, _, temp_frame = get_face_enhancer().enhance(
46+
temp_frame,
47+
paste_back=True
48+
)
49+
return temp_frame
50+
51+
52+
def process_frame(source_face: Face, temp_frame: Frame) -> Frame:
53+
target_face = get_one_face(temp_frame)
54+
if target_face:
55+
temp_frame = enhance_face(temp_frame)
56+
return temp_frame
57+
58+
59+
def process_frames(source_path: str, temp_frame_paths: List[str], progress: Any = None) -> None:
60+
for temp_frame_path in temp_frame_paths:
61+
temp_frame = cv2.imread(temp_frame_path)
62+
result = process_frame(None, temp_frame)
63+
cv2.imwrite(temp_frame_path, result)
64+
if progress:
65+
progress.update(1)
66+
67+
68+
def process_image(source_path: str, target_path: str, output_path: str) -> None:
69+
target_frame = cv2.imread(target_path)
70+
result = process_frame(None, target_frame)
71+
cv2.imwrite(output_path, result)
72+
73+
74+
def process_video(source_path: str, temp_frame_paths: List[str]) -> None:
75+
roop.processors.frame.core.process_video(None, temp_frame_paths, process_frames)

‎roop/processors/frame/face_swapper.py

+86
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
from typing import Any, List
2+
import cv2
3+
import insightface
4+
import threading
5+
6+
import roop.globals
7+
import roop.processors.frame.core
8+
from roop.core import update_status
9+
from roop.face_analyser import get_one_face, get_many_faces
10+
from roop.typing import Face, Frame
11+
from roop.utilities import conditional_download, resolve_relative_path, is_image, is_video
12+
13+
FACE_SWAPPER = None
14+
THREAD_LOCK = threading.Lock()
15+
NAME = 'ROOP.FACE-SWAPPER'
16+
17+
18+
def pre_check() -> bool:
19+
download_directory_path = resolve_relative_path('../models')
20+
conditional_download(download_directory_path, ['https://huggingface.co/henryruhs/roop/resolve/main/inswapper_128.onnx'])
21+
return True
22+
23+
24+
def pre_start() -> bool:
25+
if not is_image(roop.globals.source_path):
26+
update_status('Select an image for source path.', NAME)
27+
return False
28+
elif not get_one_face(cv2.imread(roop.globals.source_path)):
29+
update_status('No face in source path detected.', NAME)
30+
return False
31+
if not is_image(roop.globals.target_path) and not is_video(roop.globals.target_path):
32+
update_status('Select an image or video for target path.', NAME)
33+
return False
34+
return True
35+
36+
37+
def get_face_swapper() -> Any:
38+
global FACE_SWAPPER
39+
40+
with THREAD_LOCK:
41+
if FACE_SWAPPER is None:
42+
model_path = resolve_relative_path('../models/inswapper_128.onnx')
43+
FACE_SWAPPER = insightface.model_zoo.get_model(model_path, providers=roop.globals.execution_providers)
44+
return FACE_SWAPPER
45+
46+
47+
def swap_face(source_face: Face, target_face: Face, temp_frame: Frame) -> Frame:
48+
return get_face_swapper().get(temp_frame, target_face, source_face, paste_back=True)
49+
50+
51+
def process_frame(source_face: Face, temp_frame: Frame) -> Frame:
52+
if roop.globals.many_faces:
53+
many_faces = get_many_faces(temp_frame)
54+
if many_faces:
55+
for target_face in many_faces:
56+
temp_frame = swap_face(source_face, target_face, temp_frame)
57+
else:
58+
target_face = get_one_face(temp_frame)
59+
if target_face:
60+
temp_frame = swap_face(source_face, target_face, temp_frame)
61+
return temp_frame
62+
63+
64+
def process_frames(source_path: str, temp_frame_paths: List[str], progress: Any = None) -> None:
65+
source_face = get_one_face(cv2.imread(source_path))
66+
for temp_frame_path in temp_frame_paths:
67+
temp_frame = cv2.imread(temp_frame_path)
68+
try:
69+
result = process_frame(source_face, temp_frame)
70+
cv2.imwrite(temp_frame_path, result)
71+
except Exception as exception:
72+
print(exception)
73+
pass
74+
if progress:
75+
progress.update(1)
76+
77+
78+
def process_image(source_path: str, target_path: str, output_path: str) -> None:
79+
source_face = get_one_face(cv2.imread(source_path))
80+
target_frame = cv2.imread(target_path)
81+
result = process_frame(source_face, target_frame)
82+
cv2.imwrite(output_path, result)
83+
84+
85+
def process_video(source_path: str, temp_frame_paths: List[str]) -> None:
86+
roop.processors.frame.core.process_video(source_path, temp_frame_paths, process_frames)

‎roop/swapper.py

-96
This file was deleted.

‎roop/typing.py

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
from typing import Any
2+
3+
from insightface.app.common import Face
4+
import numpy
5+
6+
Face = Face
7+
Frame = numpy.ndarray[Any, Any]

‎roop/ui.json

+155
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
{
2+
"CTk": {
3+
"fg_color": ["gray95", "gray10"]
4+
},
5+
"CTkToplevel": {
6+
"fg_color": ["gray95", "gray10"]
7+
},
8+
"CTkFrame": {
9+
"corner_radius": 6,
10+
"border_width": 0,
11+
"fg_color": ["gray90", "gray13"],
12+
"top_fg_color": ["gray85", "gray16"],
13+
"border_color": ["gray65", "gray28"]
14+
},
15+
"CTkButton": {
16+
"corner_radius": 6,
17+
"border_width": 0,
18+
"fg_color": ["#3a7ebf", "#1f538d"],
19+
"hover_color": ["#325882", "#14375e"],
20+
"border_color": ["#3E454A", "#949A9F"],
21+
"text_color": ["#DCE4EE", "#DCE4EE"],
22+
"text_color_disabled": ["gray74", "gray60"]
23+
},
24+
"CTkLabel": {
25+
"corner_radius": 0,
26+
"fg_color": "transparent",
27+
"text_color": ["gray14", "gray84"]
28+
},
29+
"CTkEntry": {
30+
"corner_radius": 6,
31+
"border_width": 2,
32+
"fg_color": ["#F9F9FA", "#343638"],
33+
"border_color": ["#979DA2", "#565B5E"],
34+
"text_color": ["gray14", "gray84"],
35+
"placeholder_text_color": ["gray52", "gray62"]
36+
},
37+
"CTkCheckbox": {
38+
"corner_radius": 6,
39+
"border_width": 3,
40+
"fg_color": ["#3a7ebf", "#1f538d"],
41+
"border_color": ["#3E454A", "#949A9F"],
42+
"hover_color": ["#325882", "#14375e"],
43+
"checkmark_color": ["#DCE4EE", "gray90"],
44+
"text_color": ["gray14", "gray84"],
45+
"text_color_disabled": ["gray60", "gray45"]
46+
},
47+
"CTkSwitch": {
48+
"corner_radius": 1000,
49+
"border_width": 3,
50+
"button_length": 0,
51+
"fg_color": ["#939BA2", "#4A4D50"],
52+
"progress_color": ["#3a7ebf", "#1f538d"],
53+
"button_color": ["gray36", "#D5D9DE"],
54+
"button_hover_color": ["gray20", "gray100"],
55+
"text_color": ["gray14", "gray84"],
56+
"text_color_disabled": ["gray60", "gray45"]
57+
},
58+
"CTkRadiobutton": {
59+
"corner_radius": 1000,
60+
"border_width_checked": 6,
61+
"border_width_unchecked": 3,
62+
"fg_color": ["#3a7ebf", "#1f538d"],
63+
"border_color": ["#3E454A", "#949A9F"],
64+
"hover_color": ["#325882", "#14375e"],
65+
"text_color": ["gray14", "gray84"],
66+
"text_color_disabled": ["gray60", "gray45"]
67+
},
68+
"CTkProgressBar": {
69+
"corner_radius": 1000,
70+
"border_width": 0,
71+
"fg_color": ["#939BA2", "#4A4D50"],
72+
"progress_color": ["#3a7ebf", "#1f538d"],
73+
"border_color": ["gray", "gray"]
74+
},
75+
"CTkSlider": {
76+
"corner_radius": 1000,
77+
"button_corner_radius": 1000,
78+
"border_width": 6,
79+
"button_length": 0,
80+
"fg_color": ["#939BA2", "#4A4D50"],
81+
"progress_color": ["gray40", "#AAB0B5"],
82+
"button_color": ["#3a7ebf", "#1f538d"],
83+
"button_hover_color": ["#325882", "#14375e"]
84+
},
85+
"CTkOptionMenu": {
86+
"corner_radius": 6,
87+
"fg_color": ["#3a7ebf", "#1f538d"],
88+
"button_color": ["#325882", "#14375e"],
89+
"button_hover_color": ["#234567", "#1e2c40"],
90+
"text_color": ["#DCE4EE", "#DCE4EE"],
91+
"text_color_disabled": ["gray74", "gray60"]
92+
},
93+
"CTkComboBox": {
94+
"corner_radius": 6,
95+
"border_width": 2,
96+
"fg_color": ["#F9F9FA", "#343638"],
97+
"border_color": ["#979DA2", "#565B5E"],
98+
"button_color": ["#979DA2", "#565B5E"],
99+
"button_hover_color": ["#6E7174", "#7A848D"],
100+
"text_color": ["gray14", "gray84"],
101+
"text_color_disabled": ["gray50", "gray45"]
102+
},
103+
"CTkScrollbar": {
104+
"corner_radius": 1000,
105+
"border_spacing": 4,
106+
"fg_color": "transparent",
107+
"button_color": ["gray55", "gray41"],
108+
"button_hover_color": ["gray40", "gray53"]
109+
},
110+
"CTkSegmentedButton": {
111+
"corner_radius": 6,
112+
"border_width": 2,
113+
"fg_color": ["#979DA2", "gray29"],
114+
"selected_color": ["#3a7ebf", "#1f538d"],
115+
"selected_hover_color": ["#325882", "#14375e"],
116+
"unselected_color": ["#979DA2", "gray29"],
117+
"unselected_hover_color": ["gray70", "gray41"],
118+
"text_color": ["#DCE4EE", "#DCE4EE"],
119+
"text_color_disabled": ["gray74", "gray60"]
120+
},
121+
"CTkTextbox": {
122+
"corner_radius": 6,
123+
"border_width": 0,
124+
"fg_color": ["gray100", "gray20"],
125+
"border_color": ["#979DA2", "#565B5E"],
126+
"text_color": ["gray14", "gray84"],
127+
"scrollbar_button_color": ["gray55", "gray41"],
128+
"scrollbar_button_hover_color": ["gray40", "gray53"]
129+
},
130+
"CTkScrollableFrame": {
131+
"label_fg_color": ["gray80", "gray21"]
132+
},
133+
"DropdownMenu": {
134+
"fg_color": ["gray90", "gray20"],
135+
"hover_color": ["gray75", "gray28"],
136+
"text_color": ["gray14", "gray84"]
137+
},
138+
"CTkFont": {
139+
"macOS": {
140+
"family": "Avenir",
141+
"size": 12,
142+
"weight": "normal"
143+
},
144+
"Windows": {
145+
"family": "Corbel",
146+
"size": 12,
147+
"weight": "normal"
148+
},
149+
"Linux": {
150+
"family": "Montserrat",
151+
"size": 12,
152+
"weight": "normal"
153+
}
154+
}
155+
}

‎roop/ui.py

+187-277
Large diffs are not rendered by default.

‎roop/utilities.py

+141
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
import glob
2+
import mimetypes
3+
import os
4+
import platform
5+
import shutil
6+
import ssl
7+
import subprocess
8+
import urllib
9+
from pathlib import Path
10+
from typing import List, Any
11+
from tqdm import tqdm
12+
13+
import roop.globals
14+
15+
TEMP_FILE = 'temp.mp4'
16+
TEMP_DIRECTORY = 'temp'
17+
18+
# monkey patch ssl for mac
19+
if platform.system().lower() == 'darwin':
20+
ssl._create_default_https_context = ssl._create_unverified_context
21+
22+
23+
def run_ffmpeg(args: List[str]) -> bool:
24+
commands = ['ffmpeg', '-hide_banner', '-hwaccel', 'auto', '-loglevel', roop.globals.log_level]
25+
commands.extend(args)
26+
try:
27+
subprocess.check_output(commands, stderr=subprocess.STDOUT)
28+
return True
29+
except Exception:
30+
pass
31+
return False
32+
33+
34+
def detect_fps(target_path: str) -> float:
35+
command = ['ffprobe', '-v', 'error', '-select_streams', 'v:0', '-show_entries', 'stream=r_frame_rate', '-of', 'default=noprint_wrappers=1:nokey=1', target_path]
36+
output = subprocess.check_output(command).decode().strip().split('/')
37+
try:
38+
numerator, denominator = map(int, output)
39+
return numerator / denominator
40+
except Exception:
41+
pass
42+
return 30.0
43+
44+
45+
def extract_frames(target_path: str) -> None:
46+
temp_directory_path = get_temp_directory_path(target_path)
47+
run_ffmpeg(['-i', target_path, '-pix_fmt', 'rgb24', os.path.join(temp_directory_path, '%04d.png')])
48+
49+
50+
def create_video(target_path: str, fps: float = 30.0) -> None:
51+
temp_output_path = get_temp_output_path(target_path)
52+
temp_directory_path = get_temp_directory_path(target_path)
53+
run_ffmpeg(['-r', str(fps), '-i', os.path.join(temp_directory_path, '%04d.png'), '-c:v', roop.globals.video_encoder, '-crf', str(roop.globals.video_quality), '-pix_fmt', 'yuv420p', '-vf', 'colorspace=bt709:iall=bt601-6-625:fast=1', '-y', temp_output_path])
54+
55+
56+
def restore_audio(target_path: str, output_path: str) -> None:
57+
temp_output_path = get_temp_output_path(target_path)
58+
done = run_ffmpeg(['-i', temp_output_path, '-i', target_path, '-c:v', 'copy', '-map', '0:v:0', '-map', '1:a:0', '-y', output_path])
59+
if not done:
60+
move_temp(target_path, output_path)
61+
62+
63+
def get_temp_frame_paths(target_path: str) -> List[str]:
64+
temp_directory_path = get_temp_directory_path(target_path)
65+
return glob.glob((os.path.join(glob.escape(temp_directory_path), '*.png')))
66+
67+
68+
def get_temp_directory_path(target_path: str) -> str:
69+
target_name, _ = os.path.splitext(os.path.basename(target_path))
70+
target_directory_path = os.path.dirname(target_path)
71+
return os.path.join(target_directory_path, TEMP_DIRECTORY, target_name)
72+
73+
74+
def get_temp_output_path(target_path: str) -> str:
75+
temp_directory_path = get_temp_directory_path(target_path)
76+
return os.path.join(temp_directory_path, TEMP_FILE)
77+
78+
79+
def normalize_output_path(source_path: str, target_path: str, output_path: str) -> Any:
80+
if source_path and target_path:
81+
source_name, _ = os.path.splitext(os.path.basename(source_path))
82+
target_name, target_extension = os.path.splitext(os.path.basename(target_path))
83+
if os.path.isdir(output_path):
84+
return os.path.join(output_path, source_name + '-' + target_name + target_extension)
85+
return output_path
86+
87+
88+
def create_temp(target_path: str) -> None:
89+
temp_directory_path = get_temp_directory_path(target_path)
90+
Path(temp_directory_path).mkdir(parents=True, exist_ok=True)
91+
92+
93+
def move_temp(target_path: str, output_path: str) -> None:
94+
temp_output_path = get_temp_output_path(target_path)
95+
if os.path.isfile(temp_output_path):
96+
if os.path.isfile(output_path):
97+
os.remove(output_path)
98+
shutil.move(temp_output_path, output_path)
99+
100+
101+
def clean_temp(target_path: str) -> None:
102+
temp_directory_path = get_temp_directory_path(target_path)
103+
parent_directory_path = os.path.dirname(temp_directory_path)
104+
if not roop.globals.keep_frames and os.path.isdir(temp_directory_path):
105+
shutil.rmtree(temp_directory_path)
106+
if os.path.exists(parent_directory_path) and not os.listdir(parent_directory_path):
107+
os.rmdir(parent_directory_path)
108+
109+
110+
def has_image_extension(image_path: str) -> bool:
111+
return image_path.lower().endswith(('png', 'jpg', 'jpeg'))
112+
113+
114+
def is_image(image_path: str) -> bool:
115+
if image_path and os.path.isfile(image_path):
116+
mimetype, _ = mimetypes.guess_type(image_path)
117+
return bool(mimetype and mimetype.startswith('image/'))
118+
return False
119+
120+
121+
def is_video(video_path: str) -> bool:
122+
if video_path and os.path.isfile(video_path):
123+
mimetype, _ = mimetypes.guess_type(video_path)
124+
return bool(mimetype and mimetype.startswith('video/'))
125+
return False
126+
127+
128+
def conditional_download(download_directory_path: str, urls: List[str]) -> None:
129+
if not os.path.exists(download_directory_path):
130+
os.makedirs(download_directory_path)
131+
for url in urls:
132+
download_file_path = os.path.join(download_directory_path, os.path.basename(url))
133+
if not os.path.exists(download_file_path):
134+
request = urllib.request.urlopen(url) # type: ignore[attr-defined]
135+
total = int(request.headers.get('Content-Length', 0))
136+
with tqdm(total=total, desc='Downloading', unit='B', unit_scale=True, unit_divisor=1024) as progress:
137+
urllib.request.urlretrieve(url, download_file_path, reporthook=lambda count, block_size, total_size: progress.update(block_size)) # type: ignore[attr-defined]
138+
139+
140+
def resolve_relative_path(path: str) -> str:
141+
return os.path.abspath(os.path.join(os.path.dirname(__file__), path))

‎roop/utils.py

-72
This file was deleted.

0 commit comments

Comments
 (0)
Please sign in to comment.