無需Avatarify 無需蒙版 一鍵生成多人版 “螞蟻呀嘿“視頻

抖音上的螞蟻呀嘿火遍全網，很多小伙伴都不知道如何制作，本文拋棄繁瑣的操作，利用PaddleHub與PaddleGAN框架一鍵生成多人版的”螞蟻呀嘿“視頻，

首先我們需要安裝PaddleHub,利用其中的face detection功能來定位照片中人臉，

安裝方法如下：

pip install paddlehub --upgrade -i https://pypi.douban.com/simple

安裝之后paddlehub之后，還需要安裝一下人臉檢測的模型，命令如下:

hub install ultra_light_fast_generic_face_detector_1mb_640 

生成”螞蟻呀嘿“視頻需要用到PaddleGAN套件中的動作遷移功能，所以下一步需要安裝PaddleGAN套件，因為我修改了PaddleGAN套件部分代碼，所以這個代碼已經保存在AIStudio環境中，直接安裝就可以了，使用以下命令安裝PaddleGAN，

AIStudio 地址(推薦，可直接運行):

https://aistudio.baidu.com/aistudio/projectdetail/1285661

也可以從以下地址下載：

https://gitee.com/txyugood/PaddleGAN.git

cd PaddleGAN/
pip install -v -e .

安裝PaddleGAN依賴的PaddlePaddle框架，

python -m pip install https://paddle-wheel.bj.bcebos.com/2.0.0-rc0-gpu-cuda10.1-cudnn7-mkl_gcc8.2%2Fpaddlepaddle_gpu-2.0.0rc0.post101-cp37-cp37m-linux_x86_64.whl

在PaddleGAN/application/tools新建一個first-order-mayi.py檔案，該檔案就是生成”螞蟻呀嘿“視頻的主程式，

代碼如下：

#  Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve.
#
#Licensed under the Apache License, Version 2.0 (the "License");
#you may not use this file except in compliance with the License.
#You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
#Unless required by applicable law or agreed to in writing, software
#distributed under the License is distributed on an "AS IS" BASIS,
#WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#See the License for the specific language governing permissions and
#limitations under the License.

import argparse

import os
import paddle
from ppgan.apps.first_order_predictor import FirstOrderPredictor
from skimage import img_as_ubyte
import paddlehub as hub
import math
import cv2
import imageio

parser = argparse.ArgumentParser()
parser.add_argument("--config", default=None, help="path to config")
parser.add_argument("--weight_path",
                    default=None,
                    help="path to checkpoint to restore")
parser.add_argument("--source_image", type=str, help="path to source image")
parser.add_argument("--driving_video", type=str, help="path to driving video")
parser.add_argument("--output", default='output', help="path to output")
parser.add_argument("--relative",
                    dest="relative",
                    action="store_true",
                    help="use relative or absolute keypoint coordinates")
parser.add_argument(
    "--adapt_scale",
    dest="adapt_scale",
    action="store_true",
    help="adapt movement scale based on convex hull of keypoints")

parser.add_argument(
    "--find_best_frame",
    dest="find_best_frame",
    action="store_true",
    help=
    "Generate from the frame that is the most alligned with source. (Only for faces, requires face_aligment lib)"
)

parser.add_argument("--best_frame",
                    dest="best_frame",
                    type=int,
                    default=None,
                    help="Set frame to start from.")
parser.add_argument("--cpu", dest="cpu", action="store_true", help="cpu mode.")

parser.set_defaults(relative=False)
parser.set_defaults(adapt_scale=False)

if __name__ == "__main__":
    #決議引數
    args = parser.parse_args()
    if args.cpu:
        paddle.set_device('cpu')
    cache_path = os.path.join(args.output,"cache")
    if not os.path.exists(cache_path):
        os.makedirs(cache_path)
    image_path = args.source_image
    
    origin_img = cv2.imread(image_path)
    image_width = origin_img.shape[1]
    image_hegiht = origin_img.shape[0]
	#獲取人臉檢測模型
    module = hub.Module(name="ultra_light_fast_generic_face_detector_1mb_640")
    face_detecions = module.face_detection(paths = [image_path], visualization=True, output_dir='face_detection_output')
    face_detecions = face_detecions[0]['data']
    
    face_list = []
    #遍歷人臉檢測結果，并保存人臉部分的圖片，在原圖中的位置，以及人臉的尺寸，
    #這里需要對檢測結果得出的尺寸近一步放大，
    for i, face_dect in enumerate(face_detecions):
        left = math.ceil(face_dect['left'])
        right = math.ceil(face_dect['right'])
        top = math.ceil(face_dect['top'])
        bottom = math.ceil(face_dect['bottom'])
        width = right - left
        height = bottom - top
        center_w = left + width // 2
        center_h = top + height // 2

        new_left = max(center_w - height, 0)
        new_right = min(center_w + height, image_width)

        new_top = max(center_h - height, 0)
        new_bottom = min(center_h + height, image_hegiht)

        origin_img = cv2.imread(image_path)
        face_img = origin_img[new_top:new_bottom, new_left:new_right, :]
        face_height = face_img.shape[0]
        face_weight = face_img.shape[1]

        cv2.imwrite(os.path.join(cache_path,'face_{}.jpeg'.format(i)), face_img)

        face_list.append({"path" : os.path.join(cache_path,'face_{}.jpeg'.format(i)),
                            "width":face_weight, "height":face_height, 
                            "top":new_top, "bottom":new_bottom,
                            "left":new_left, "right":new_right})
    #使用驅動視頻，對所有的人臉圖片進行動作遷移，將生產的圖片序列保存起來，
    frames = 0
    for face_dict in face_list:
        predictor = FirstOrderPredictor(output=args.output,
                                        weight_path=args.weight_path,
                                        config=args.config,
                                        relative=args.relative,
                                        adapt_scale=args.adapt_scale,
                                        find_best_frame=args.find_best_frame,
                                        best_frame=args.best_frame)
        predictions,fps = predictor.run(face_dict["path"], args.driving_video)
        face_dict['pre'] = predictions
        frames = len(predictions)
    images = []    
    #遍歷所有的生成的圖片序列，放到原圖中對應的位置，
    for i in range(frames):
        new_frame = origin_img.copy()
        new_frame = new_frame[:,:,[2,1,0]]
        for face_dict in face_list:
            pre = face_dict["pre"][i]
            face_weight = face_dict["width"]
            face_height = face_dict["height"]
            top = face_dict["top"]
            bottom = face_dict["bottom"]
            left = face_dict["left"]
            right = face_dict["right"]
            img = cv2.resize(pre,(face_weight, face_height))
            new_frame[top:bottom, left:right, :] = img_as_ubyte(img)
        images.append(new_frame)
    #生成視頻，這一步是沒有聲音的，后面是用ffmpeg合成帶音頻的視頻檔案，
    imageio.mimsave(os.path.join(args.output, 'result.mp4'),
                [img_as_ubyte(frame) for frame in images],
                fps=fps)
    #使用ffmpeg將聲音合并到視頻中去，
    os.system("ffmpeg -i"  + os.path.join(args.output, 'result.mp4') + "-i /home/aistudio/MYYH.mp3 -c:v copy -c:a aac -strict experimental " + os.path.join(args.output, 'result.mp4'))

運行腳本生成視頻，

此處借用了GT大佬
https://aistudio.baidu.com/aistudio/projectdetail/1584416專案中的驅動視頻，

/home/aistudio/1.jpeg是測驗的照片，可以使用右側的上傳功能上傳自己的照片，然后替換–source_image 后面的路徑后，運行腳本即可，

最終/home/aistudio/output/mayiyahei.mp4就是最終生成的"螞蟻呀嘿"視頻，

cd /home/aistudio/PaddleGAN/applications/
python -u tools/first-order-mayi.py  \
     --driving_video /home/aistudio/MaYiYaHei.mp4 \
     --source_image /home/aistudio/1.jpeg \
     --relative --adapt_scale \
     --output /home/aistudio/output

最后放一張效果圖:

該程式目前還有許多可以改進的地方，后續會繼續優化，

推薦使用AI Studio運行該程式，不但有免費的V100算力可使用，還可以方便的一鍵運行腳本生成視頻，

歡迎關注我的公眾號:人工智能研習社，分享更多的人工智能技術干貨，