前言

終于到了這一步了，看了很久很久的slowfast，這次終于用slowfast訓練了自己的資料集（只是用了個非常小的，非常小的資料集跑了一下），并且格式修改程序是手動修改的，訓練程序所需要的yaml檔案也是手動寫出來的（后面會改成程式自動修改）

在看這篇訓練自己的資料集博客之前，我覺得有必要了解一下之前我寫一些博客：
的slowfast的訓練，slowfast的減少資料集

1：【SlowFast復現】SlowFast Networks for Video Recognition復現代碼 使用自己的視頻進行demo檢測

2:【Faster RCNN & detectron2】detectron2實作Faster RCNN目標檢測

3，【faster rcnn 實作via的自動框人】使用detectron2中faster rcnn 演算法生成人的坐標，將坐標匯入via(VGG Image Annotator)中，實作自動框選出人的區域

4，【ffmpeg裁剪視頻faster rcnn自動檢測 via】全自動實作ffmpeg將視頻切割為圖片幀，再使用faster rcnn將圖片中的人檢測出來，最后將檢測結果轉化為via可識別的csv格式

5，【slowfast復現 訓練】訓練程序 制作ava資料集 復現 SlowFast Networks for Video Recognition 訓練 train

6，【slowfast 減少ava資料集】將ava資料集縮小到2個，對資料集做訓練，然后進行檢測，為訓練自己的資料集做準備

一，視頻的處理

這里使用的資料集就是自己定義的資料集了，大家做什么領域的就用什么視頻，比如有人要做手勢識別的，就用手勢的，有人要做人摔倒的，那就找人摔倒的視頻，

我這里就用一個簡單的視頻，人在說話的視頻，視頻不多，就2個3秒的視頻（這里是我用來做例子的，如果真實情況，2個視頻3秒視頻是遠遠不夠的），

1.1 視頻準備

準備2個3秒的視頻，最好里面人少一點，減少標注的難度，下圖是我選擇的幾個3秒的視頻（為了快速，我這里只選擇其中2個）

下面是使用ffmpeg將一段長視頻裁剪為3秒視頻的命令，

ffmpeg -ss 00:00:00.0 -to 00:00:3.0 -i "../cutVideo2/1/1.mp4" "../shortVideoTrain/A.mp4"
ffmpeg -ss 00:00:03.0 -to 00:00:06.0 -i "../cutVideo2/1/1.mp4" "../shortVideoTrain/B.mp4"
ffmpeg -ss 00:00:06.0 -to 00:00:09.0 -i "../cutVideo2/1/1.mp4" "../shortVideoTrain/C.mp4"
ffmpeg -ss 00:00:09.0 -to 00:00:12.0 -i "../cutVideo2/1/1.mp4" "../shortVideoTrain/D.mp4"
ffmpeg -ss 00:00:12.0 -to 00:00:15.0 -i "../cutVideo2/1/1.mp4" "../shortVideoTrain/E.mp4"

1.2 切割視頻為圖片

這里需要把3秒的視頻按2種方式切割

第一種是，把視頻按照每秒1幀來裁剪，這樣裁剪的目的是用來標注，因為ava資料集就是1秒1幀的標志，

命令如下：

#切割圖片，每秒1幀
IN_DATA_DIR="../shortVideoTrain/"
OUT_DATA_DIR="../shortVideoTrainCut/"

if [[ ! -d "${OUT_DATA_DIR}" ]]; then
  echo "${OUT_DATA_DIR} doesn't exist. Creating it.";
  mkdir -p ${OUT_DATA_DIR}
fi

for video in $(ls -A1 -U ${IN_DATA_DIR}/*)
do
  video_name=${video##*/}

  if [[ $video_name = *".webm" ]]; then
    video_name=${video_name::-5}
  else
    video_name=${video_name::-4}
  fi

  out_video_dir=${OUT_DATA_DIR}/${video_name}/
  mkdir -p "${out_video_dir}"

  out_name="${out_video_dir}/${video_name}_%06d.jpg"

  ffmpeg -i "${video}" -r 1 -q:v 1 "${out_name}"
done

切割結果如下（3秒的視頻被切割為5張）：

第二種是，把視頻按照每秒30幀來裁剪，這樣裁剪的目的是，用于slowfast的訓練，因為slowfast，在slow流里，1秒會采集到15幀，在fast流里，1秒會采集到2幀，

命令如下：

#切割圖片，每秒30幀
IN_DATA_DIR="../shortVideoTrain/"
OUT_DATA_DIR="../shortVideoFrames/"

if [[ ! -d "${OUT_DATA_DIR}" ]]; then
  echo "${OUT_DATA_DIR} doesn't exist. Creating it.";
  mkdir -p ${OUT_DATA_DIR}
fi

for video in $(ls -A1 -U ${IN_DATA_DIR}/*)
do
  video_name=${video##*/}

  if [[ $video_name = *".webm" ]]; then
    video_name=${video_name::-5}
  else
    video_name=${video_name::-4}
  fi

  out_video_dir=${OUT_DATA_DIR}/${video_name}/
  mkdir -p "${out_video_dir}"

  out_name="${out_video_dir}/${video_name}_%06d.jpg"

  ffmpeg -i "${video}" -r 30 -q:v 1 "${out_name}"
done

切割結果如下（3秒的視頻被切割為90張）：

1.3 使用faster rcnn自動框人

其中這一步可以自己框人，因為2個視頻就10張圖片，但是，之后的資料集，應該是上萬張，而且一張圖片可能多達40個人，自己手動標注會很累的，所以這里采用我之前寫的博客里的方法，使用detectron2里的faster rcnn 實作人的自動框出，

【faster rcnn 實作via的自動框人】使用detectron2中faster rcnn 演算法生成人的坐標，將坐標匯入via(VGG Image Annotator)中，實作自動框選出人的區域

【ffmpeg裁剪視頻faster rcnn自動檢測 via】全自動實作ffmpeg將視頻切割為圖片幀，再使用faster rcnn將圖片中的人檢測出來，最后將檢測結果轉化為via可識別的csv格式

這里就不說怎么自動標注了，上面兩篇博客寫的很清楚自動標注的程序，

自動標注后，會有生成一個csv檔案，如下圖所示：

打開這個檔案，把里面的 單引號 ’ 全部刪掉（可以替換為空），結果如下：

filename,file_size,file_attributes,region_count,region_id,region_shape_attributes,region_attributes
A_000001.jpg,33338,"""{}""",3,0,"{""name"": ""rect"", ""x"": 285, ""y"": 93, ""width"": 108, ""height"": 284}","""{}"""
A_000001.jpg,33338,"""{}""",3,1,"{""name"": ""rect"", ""x"": 537, ""y"": 357, ""width"": 50, ""height"": 46}","""{}"""
A_000001.jpg,33338,"""{}""",3,2,"{""name"": ""rect"", ""x"": 238, ""y"": 361, ""width"": 52, ""height"": 42}","""{}"""
A_000002.jpg,33430,"""{}""",3,0,"{""name"": ""rect"", ""x"": 535, ""y"": 361, ""width"": 55, ""height"": 42}","""{}"""
A_000002.jpg,33430,"""{}""",3,1,"{""name"": ""rect"", ""x"": 285, ""y"": 95, ""width"": 106, ""height"": 268}","""{}"""
A_000002.jpg,33430,"""{}""",3,2,"{""name"": ""rect"", ""x"": 238, ""y"": 361, ""width"": 52, ""height"": 42}","""{}"""
A_000003.jpg,33416,"""{}""",3,0,"{""name"": ""rect"", ""x"": 328, ""y"": 96, ""width"": 105, ""height"": 152}","""{}"""
A_000003.jpg,33416,"""{}""",3,1,"{""name"": ""rect"", ""x"": 234, ""y"": 360, ""width"": 52, ""height"": 43}","""{}"""
A_000003.jpg,33416,"""{}""",3,2,"{""name"": ""rect"", ""x"": 543, ""y"": 367, ""width"": 52, ""height"": 36}","""{}"""
A_000004.jpg,34544,"""{}""",3,0,"{""name"": ""rect"", ""x"": 328, ""y"": 94, ""width"": 110, ""height"": 152}","""{}"""
A_000004.jpg,34544,"""{}""",3,1,"{""name"": ""rect"", ""x"": 232, ""y"": 363, ""width"": 52, ""height"": 40}","""{}"""
A_000004.jpg,34544,"""{}""",3,2,"{""name"": ""rect"", ""x"": 541, ""y"": 365, ""width"": 52, ""height"": 38}","""{}"""
A_000005.jpg,34433,"""{}""",4,0,"{""name"": ""rect"", ""x"": 328, ""y"": 93, ""width"": 106, ""height"": 155}","""{}"""
A_000005.jpg,34433,"""{}""",4,1,"{""name"": ""rect"", ""x"": 233, ""y"": 364, ""width"": 53, ""height"": 39}","""{}"""
A_000005.jpg,34433,"""{}""",4,2,"{""name"": ""rect"", ""x"": 367, ""y"": 354, ""width"": 54, ""height"": 48}","""{}"""
A_000005.jpg,34433,"""{}""",4,3,"{""name"": ""rect"", ""x"": 543, ""y"": 369, ""width"": 52, ""height"": 34}","""{}"""

1.4 via標注圖片

打開via，匯入圖片，匯入csv，結果如下：

接下來，要做的是把一些沒用的框給去掉，比如說上面圖片中，只有腦袋的檢測結果，這些就是不需要的，去掉，去掉后結果如下：

注意，這里不打標簽，不打標簽，因為slowfast所需要的標簽格式和via的標簽格式不同，這里在后面會手動填入標簽，在我之后的博客里，會把via和slowfast的資料集格式的轉換博客補上，

我們處理好上面所說的之后，匯出csv檔案，如下圖：

filename,file_size,file_attributes,region_count,region_id,region_shape_attributes,region_attributes
A_000001.jpg,33338,"{""0"":""{"",""1"":""}""}",1,0,"{""name"":""rect"",""x"":285,""y"":93,""width"":108,""height"":284}","{""0"":""{"",""1"":""}""}"
A_000002.jpg,33430,"{""0"":""{"",""1"":""}""}",1,0,"{""name"":""rect"",""x"":285,""y"":95,""width"":106,""height"":268}","{""0"":""{"",""1"":""}""}"
A_000003.jpg,33416,"{""0"":""{"",""1"":""}""}",1,0,"{""name"":""rect"",""x"":328,""y"":96,""width"":105,""height"":152}","{""0"":""{"",""1"":""}""}"
A_000004.jpg,34544,"{""0"":""{"",""1"":""}""}",1,0,"{""name"":""rect"",""x"":328,""y"":94,""width"":110,""height"":152}","{""0"":""{"",""1"":""}""}"
A_000005.jpg,34433,"{""0"":""{"",""1"":""}""}",1,0,"{""name"":""rect"",""x"":328,""y"":93,""width"":106,""height"":155}","{""0"":""{"",""1"":""}""}"

二，資料集檔案

在第一部分，我們已經拿到了2個圖片檔案的標注csv，接下來我們就手動把資料集的相關檔案寫了，

2.1 資料集檔案總覽

annotations
—person_box_67091280_iou90
——ava_detection_train_boxes_and_labels_include_negative_v2.2.csv
——ava_detection_val_boxes_and_labels.csv
—ava_action_list_v2.2_for_activitynet_2019.pbtxt
—ava_val_excluded_timestamps_v2.2.csv
—ava_train_v2.2.csv
—ava_val_v2.2.csv

frame_lists
—train.csv
—val.csv

frames
—A
——A_000001.jpg
——A_0000012.jpg
…
——A_000090.jpg
—B
——B_000001.jpg
——B_0000012.jpg
…
——B_000090.jpg

2.2 annotations

2.2.1 ava_train_v2.2.csv

A,1,0.395,0.230,0.545,0.933,1,0
A,2,0.395,0.235,0.543,0.898,1,0
A,3,0.455,0.237,0.601,0.613,1,0
A,4,0.455,0.232,0.608,0.608,1,0
A,5,0.455,0.230,0.602,0.613,1,0

2.2.2 ava_val_v2.2.csv

B,1,0.455,0.227,0.601,0.613,1,0
B,2,0.455,0.227,0.601,0.611,1,0
B,3,0.455,0.225,0.601,0.611,1,0
B,4,0.455,0.230,0.602,0.616,1,0
B,5,0.455,0.227,0.604,0.611,1,0

2.2.3 ava_val_excluded_timestamps_v2.2.csv

這是空檔案

2.2.4 ava_action_list_v2.2_for_activitynet_2019.pbtxt

item {
  name: "talk"
  id: 1
}

2.2.5 ava_detection_train_boxes_and_labels_include_negative_v2.2.csv

A,1,0.395,0.230,0.545,0.933,1,0.996382
A,2,0.395,0.235,0.543,0.898,1,0.996382
A,3,0.455,0.237,0.601,0.613,1,0.996382
A,4,0.455,0.232,0.608,0.608,1,0.996382
A,5,0.455,0.230,0.602,0.613,1,0.996382

2.2.6 ava_detection_val_boxes_and_labels.csv

B,1,0.455,0.227,0.601,0.613,,0.995518
B,2,0.455,0.227,0.601,0.611,,0.995518
B,3,0.455,0.225,0.601,0.611,,0.995518
B,4,0.455,0.230,0.602,0.616,,0.995518
B,5,0.455,0.227,0.604,0.611,,0.995518

2.3 frame_lists

2.3.1 train.csv

original_vido_id video_id frame_id path labels
A 0 0 A/A_000001.jpg ""
A 0 1 A/A_000002.jpg ""
A 0 2 A/A_000003.jpg ""
A 0 3 A/A_000004.jpg ""
A 0 4 A/A_000005.jpg ""
A 0 5 A/A_000006.jpg ""
A 0 6 A/A_000007.jpg ""
A 0 7 A/A_000008.jpg ""
A 0 8 A/A_000009.jpg ""
A 0 9 A/A_000010.jpg ""
A 0 10 A/A_000011.jpg ""
A 0 11 A/A_000012.jpg ""
A 0 12 A/A_000013.jpg ""
A 0 13 A/A_000014.jpg ""
A 0 14 A/A_000015.jpg ""
A 0 15 A/A_000016.jpg ""
A 0 16 A/A_000017.jpg ""
A 0 17 A/A_000018.jpg ""
A 0 18 A/A_000019.jpg ""
A 0 19 A/A_000020.jpg ""
A 0 20 A/A_000021.jpg ""
A 0 21 A/A_000022.jpg ""
A 0 22 A/A_000023.jpg ""
A 0 23 A/A_000024.jpg ""
A 0 24 A/A_000025.jpg ""
A 0 25 A/A_000026.jpg ""
A 0 26 A/A_000027.jpg ""
A 0 27 A/A_000028.jpg ""
A 0 28 A/A_000029.jpg ""
A 0 29 A/A_000030.jpg ""
A 0 30 A/A_000031.jpg ""
A 0 31 A/A_000032.jpg ""
A 0 32 A/A_000033.jpg ""
A 0 33 A/A_000034.jpg ""
A 0 34 A/A_000035.jpg ""
A 0 35 A/A_000036.jpg ""
A 0 36 A/A_000037.jpg ""
A 0 37 A/A_000038.jpg ""
A 0 38 A/A_000039.jpg ""
A 0 39 A/A_000040.jpg ""
A 0 40 A/A_000041.jpg ""
A 0 41 A/A_000042.jpg ""
A 0 42 A/A_000043.jpg ""
A 0 43 A/A_000044.jpg ""
A 0 44 A/A_000045.jpg ""
A 0 45 A/A_000046.jpg ""
A 0 46 A/A_000047.jpg ""
A 0 47 A/A_000048.jpg ""
A 0 48 A/A_000049.jpg ""
A 0 49 A/A_000050.jpg ""
A 0 50 A/A_000051.jpg ""
A 0 51 A/A_000052.jpg ""
A 0 52 A/A_000053.jpg ""
A 0 53 A/A_000054.jpg ""
A 0 54 A/A_000055.jpg ""
A 0 55 A/A_000056.jpg ""
A 0 56 A/A_000057.jpg ""
A 0 57 A/A_000058.jpg ""
A 0 58 A/A_000059.jpg ""
A 0 59 A/A_000060.jpg ""
A 0 60 A/A_000061.jpg ""
A 0 61 A/A_000062.jpg ""
A 0 62 A/A_000063.jpg ""
A 0 63 A/A_000064.jpg ""
A 0 64 A/A_000065.jpg ""
A 0 65 A/A_000066.jpg ""
A 0 66 A/A_000067.jpg ""
A 0 67 A/A_000068.jpg ""
A 0 68 A/A_000069.jpg ""
A 0 69 A/A_000070.jpg ""
A 0 70 A/A_000071.jpg ""
A 0 71 A/A_000072.jpg ""
A 0 72 A/A_000073.jpg ""
A 0 73 A/A_000074.jpg ""
A 0 74 A/A_000075.jpg ""
A 0 75 A/A_000076.jpg ""
A 0 76 A/A_000077.jpg ""
A 0 77 A/A_000078.jpg ""
A 0 78 A/A_000079.jpg ""
A 0 79 A/A_000080.jpg ""
A 0 80 A/A_000081.jpg ""
A 0 81 A/A_000082.jpg ""
A 0 82 A/A_000083.jpg ""
A 0 83 A/A_000084.jpg ""
A 0 84 A/A_000085.jpg ""
A 0 85 A/A_000086.jpg ""
A 0 86 A/A_000087.jpg ""
A 0 87 A/A_000088.jpg ""
A 0 88 A/A_000089.jpg ""
A 0 89 A/A_000090.jpg ""

2.3.2 val.csv

originBl_vido_id video_id frBme_id pBth lBbels
B 1 0 B/B_000001.jpg ""
B 1 1 B/B_000002.jpg ""
B 1 2 B/B_000003.jpg ""
B 1 3 B/B_000004.jpg ""
B 1 4 B/B_000005.jpg ""
B 1 5 B/B_000006.jpg ""
B 1 6 B/B_000007.jpg ""
B 1 7 B/B_000008.jpg ""
B 1 8 B/B_000009.jpg ""
B 1 9 B/B_000010.jpg ""
B 1 10 B/B_000011.jpg ""
B 1 11 B/B_000012.jpg ""
B 1 12 B/B_000013.jpg ""
B 1 13 B/B_000014.jpg ""
B 1 14 B/B_000015.jpg ""
B 1 15 B/B_000016.jpg ""
B 1 16 B/B_000017.jpg ""
B 1 17 B/B_000018.jpg ""
B 1 18 B/B_000019.jpg ""
B 1 19 B/B_000020.jpg ""
B 1 20 B/B_000021.jpg ""
B 1 21 B/B_000022.jpg ""
B 1 22 B/B_000023.jpg ""
B 1 23 B/B_000024.jpg ""
B 1 24 B/B_000025.jpg ""
B 1 25 B/B_000026.jpg ""
B 1 26 B/B_000027.jpg ""
B 1 27 B/B_000028.jpg ""
B 1 28 B/B_000029.jpg ""
B 1 29 B/B_000030.jpg ""
B 1 30 B/B_000031.jpg ""
B 1 31 B/B_000032.jpg ""
B 1 32 B/B_000033.jpg ""
B 1 33 B/B_000034.jpg ""
B 1 34 B/B_000035.jpg ""
B 1 35 B/B_000036.jpg ""
B 1 36 B/B_000037.jpg ""
B 1 37 B/B_000038.jpg ""
B 1 38 B/B_000039.jpg ""
B 1 39 B/B_000040.jpg ""
B 1 40 B/B_000041.jpg ""
B 1 41 B/B_000042.jpg ""
B 1 42 B/B_000043.jpg ""
B 1 43 B/B_000044.jpg ""
B 1 44 B/B_000045.jpg ""
B 1 45 B/B_000046.jpg ""
B 1 46 B/B_000047.jpg ""
B 1 47 B/B_000048.jpg ""
B 1 48 B/B_000049.jpg ""
B 1 49 B/B_000050.jpg ""
B 1 50 B/B_000051.jpg ""
B 1 51 B/B_000052.jpg ""
B 1 52 B/B_000053.jpg ""
B 1 53 B/B_000054.jpg ""
B 1 54 B/B_000055.jpg ""
B 1 55 B/B_000056.jpg ""
B 1 56 B/B_000057.jpg ""
B 1 57 B/B_000058.jpg ""
B 1 58 B/B_000059.jpg ""
B 1 59 B/B_000060.jpg ""
B 1 60 B/B_000061.jpg ""
B 1 61 B/B_000062.jpg ""
B 1 62 B/B_000063.jpg ""
B 1 63 B/B_000064.jpg ""
B 1 64 B/B_000065.jpg ""
B 1 65 B/B_000066.jpg ""
B 1 66 B/B_000067.jpg ""
B 1 67 B/B_000068.jpg ""
B 1 68 B/B_000069.jpg ""
B 1 69 B/B_000070.jpg ""
B 1 70 B/B_000071.jpg ""
B 1 71 B/B_000072.jpg ""
B 1 72 B/B_000073.jpg ""
B 1 73 B/B_000074.jpg ""
B 1 74 B/B_000075.jpg ""
B 1 75 B/B_000076.jpg ""
B 1 76 B/B_000077.jpg ""
B 1 77 B/B_000078.jpg ""
B 1 78 B/B_000079.jpg ""
B 1 79 B/B_000080.jpg ""
B 1 80 B/B_000081.jpg ""
B 1 81 B/B_000082.jpg ""
B 1 82 B/B_000083.jpg ""
B 1 83 B/B_000084.jpg ""
B 1 84 B/B_000085.jpg ""
B 1 85 B/B_000086.jpg ""
B 1 86 B/B_000087.jpg ""
B 1 87 B/B_000088.jpg ""
B 1 88 B/B_000089.jpg ""
B 1 89 B/B_000090.jpg ""

2.4 frames

這里的存放我們在第一部分切割的圖片，1秒切割30張的那個檔案：

三，slowfast 訓練自己的資料集

在第一部分和第二部分，我們把需要準備的東西都準備好了，下一步就是在slowfast修改訓練所需要的代碼，

3.1 預訓練模型

最好使用預訓練模型，這樣可以縮短訓練的一個時間，我用的預訓練模型如下圖

模型下載官網，預訓練模型下載鏈接

下載下來后，放在檔案夾/SlowFast/configs/AVA/c2/下面，如下圖：

3.2 組態檔

在/SlowFast/configs/AVA/下創建一個新的yaml檔案：SLOWFAST_32x2_R50_SHORT5.yaml，如下圖：

代碼如下：

TRAIN:
  ENABLE: True
  DATASET: ava
  BATCH_SIZE: 8 #64
  EVAL_PERIOD: 5
  CHECKPOINT_PERIOD: 1
  AUTO_RESUME: True
  CHECKPOINT_FILE_PATH: '/home/lxn/0yangfan/Slowfast2/SlowFast-master/configs/AVA/c2/SLOWFAST_32x2_R101_50_50.pkl'  #path to pretrain model
  CHECKPOINT_TYPE: caffe2
DATA:
  NUM_FRAMES: 32
  SAMPLING_RATE: 2
  TRAIN_JITTER_SCALES: [256, 320]
  TRAIN_CROP_SIZE: 224
  TEST_CROP_SIZE: 224
  INPUT_CHANNEL_NUM: [3, 3]
  PATH_TO_DATA_DIR: '/home/lxn/yf_videos/myCustomava'
DETECTION:
  ENABLE: True
  ALIGNED: True
AVA:
  FRAME_DIR: '/home/lxn/yf_videos/myCustomava/frames'
  FRAME_LIST_DIR: '/home/lxn/yf_videos/myCustomava/frame_lists'
  ANNOTATION_DIR: '/home/lxn/yf_videos/myCustomava/annotations'
  DETECTION_SCORE_THRESH: 0.8
  TRAIN_PREDICT_BOX_LISTS: [
    "ava_train_v2.2.csv",
    "person_box_67091280_iou90/ava_detection_train_boxes_and_labels_include_negative_v2.2.csv",
  ]
  TEST_PREDICT_BOX_LISTS: ["person_box_67091280_iou90/ava_detection_val_boxes_and_labels.csv"]
  
  
SLOWFAST:
  ALPHA: 4
  BETA_INV: 8
  FUSION_CONV_CHANNEL_RATIO: 2
  FUSION_KERNEL_SZ: 7
RESNET:
  ZERO_INIT_FINAL_BN: True
  WIDTH_PER_GROUP: 64
  NUM_GROUPS: 1
  DEPTH: 50
  TRANS_FUNC: bottleneck_transform
  STRIDE_1X1: False
  NUM_BLOCK_TEMP_KERNEL: [[3, 3], [4, 4], [6, 6], [3, 3]]
  SPATIAL_DILATIONS: [[1, 1], [1, 1], [1, 1], [2, 2]]
  SPATIAL_STRIDES: [[1, 1], [2, 2], [2, 2], [1, 1]]
NONLOCAL:
  LOCATION: [[[], []], [[], []], [[], []], [[], []]]
  GROUP: [[1, 1], [1, 1], [1, 1], [1, 1]]
  INSTANTIATION: dot_product
  POOL: [[[1, 2, 2], [1, 2, 2]], [[1, 2, 2], [1, 2, 2]], [[1, 2, 2], [1, 2, 2]], [[1, 2, 2], [1, 2, 2]]]
BN:
  USE_PRECISE_STATS: False
  NUM_BATCHES_PRECISE: 200
SOLVER:
  BASE_LR: 0.1
  LR_POLICY: steps_with_relative_lrs
  STEPS: [0, 10, 15, 20]
  LRS: [1, 0.1, 0.01, 0.001]
  MAX_EPOCH: 20
  MOMENTUM: 0.9
  WEIGHT_DECAY: 1e-7
  WARMUP_EPOCHS: 5.0
  WARMUP_START_LR: 0.000125
  OPTIMIZING_METHOD: sgd
MODEL:
  NUM_CLASSES: 1
  ARCH: slowfast
  MODEL_NAME: SlowFast
  LOSS_FUNC: bce
  DROPOUT_RATE: 0.5
  HEAD_ACT: sigmoid
TEST:
  ENABLE: False
  DATASET: ava
  BATCH_SIZE: 8
DATA_LOADER:
  NUM_WORKERS: 2
  PIN_MEMORY: True
NUM_GPUS: 1
NUM_SHARDS: 1
RNG_SEED: 0
OUTPUT_DIR: .

有幾個需要注意的地方：

1，TRAIN:CHECKPOINT_FILE_PATH 就是我們下載的與訓練模型的位置
2，DATA:PATH_TO_DATA_DIR 就是我們第二部分制作的資料集檔案
3，AVA: 下面的路徑也是對應第二部分資料集檔案對應的地方
4，MODEL:NUM_CLASSES: 1 這里是最需要主義的，這里classes必需為1，因為我們只有talk這一個分類，

3.3 訓練程序

 python tools/run_net.py --cfg configs/AVA/SLOWFAST_32x2_R50_SHORT5.yaml

四 檢測模型效果

使用slowfast檢測自己的視頻可以參考我之前寫的一篇博客：

【SlowFast復現】SlowFast Networks for Video Recognition復現代碼 使用自己的視頻進行demo檢測

詳細程序我就不寫了，上面這篇博客寫的有，

4.2 創建json檔案

在目錄SlowFast-master/demo/AVA/下創建一個json檔案：ava2.json

{"talk": 0}

4.2 創建yaml檔案

在目錄SlowFast-master/demo/AVA/下創建一個yaml檔案：SLOWFAST_32x2_R101_50_50s2.yaml

代碼如下：

TRAIN:
  ENABLE: False
  DATASET: ava
  BATCH_SIZE: 16
  EVAL_PERIOD: 1
  CHECKPOINT_PERIOD: 1
  AUTO_RESUME: True
  CHECKPOINT_FILE_PATH: '/home/lxn/0yangfan/Slowfast2/SlowFast-master/configs/AVA/c2/SLOWFAST_32x2_R101_50_50.pkl'  #path to pretrain model
  CHECKPOINT_TYPE: pytorch
DATA:
  NUM_FRAMES: 32
  SAMPLING_RATE: 2
  TRAIN_JITTER_SCALES: [256, 320]
  TRAIN_CROP_SIZE: 224
  TEST_CROP_SIZE: 256
  INPUT_CHANNEL_NUM: [3, 3]
DETECTION:
  ENABLE: True
  ALIGNED: False
AVA:
  BGR: False
  DETECTION_SCORE_THRESH: 0.8
  TEST_PREDICT_BOX_LISTS: ["person_box_67091280_iou90/ava_detection_val_boxes_and_labels.csv"]
SLOWFAST:
  ALPHA: 4
  BETA_INV: 8
  FUSION_CONV_CHANNEL_RATIO: 2
  FUSION_KERNEL_SZ: 5
RESNET:
  ZERO_INIT_FINAL_BN: True
  WIDTH_PER_GROUP: 64
  NUM_GROUPS: 1
  DEPTH: 101
  TRANS_FUNC: bottleneck_transform
  STRIDE_1X1: False
  NUM_BLOCK_TEMP_KERNEL: [[3, 3], [4, 4], [6, 6], [3, 3]]
  SPATIAL_DILATIONS: [[1, 1], [1, 1], [1, 1], [2, 2]]
  SPATIAL_STRIDES: [[1, 1], [2, 2], [2, 2], [1, 1]]
NONLOCAL:
  LOCATION: [[[], []], [[], []], [[6, 13, 20], []], [[], []]]
  GROUP: [[1, 1], [1, 1], [1, 1], [1, 1]]
  INSTANTIATION: dot_product
  POOL: [[[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]]]
BN:
  USE_PRECISE_STATS: False
  NUM_BATCHES_PRECISE: 200
SOLVER:
  MOMENTUM: 0.9
  WEIGHT_DECAY: 1e-7
  OPTIMIZING_METHOD: sgd
MODEL:
  NUM_CLASSES: 1
  ARCH: slowfast
  MODEL_NAME: SlowFast
  LOSS_FUNC: bce
  DROPOUT_RATE: 0.5
  HEAD_ACT: sigmoid
TEST:
  ENABLE: False
  DATASET: ava
  BATCH_SIZE: 8
DATA_LOADER:
  NUM_WORKERS: 2
  PIN_MEMORY: True

NUM_GPUS: 1
NUM_SHARDS: 1
RNG_SEED: 0
OUTPUT_DIR: .
#TENSORBOARD:
#  MODEL_VIS:
#    TOPK: 2
DEMO:
  ENABLE: True
  LABEL_FILE_PATH: "/home/lxn/0yangfan/Slowfast2/SlowFast-master/demo/AVA/ava2.json"
  INPUT_VIDEO: "/home/lxn/yf_videos/myVideo15mins/eight.mp4"
  OUTPUT_FILE: "/home/lxn/0yangfan/Slowfast2/SlowFast-master/Voutput/eight2.mp4"

  DETECTRON2_CFG: "COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml"
  DETECTRON2_WEIGHTS: detectron2://COCO-Detection/faster_rcnn_R_50_FPN_3x/137849458/model_final_280758.pkl

有幾個地方需要注意：

1，TRAIN:CHECKPOINT_FILE_PATH 就是第三部分訓練自己資料集的模型位置，
2，MODEL:NUM_CLASSES: 1 這里我們只有1個動作，所以只有一個分類

4.3 修改ava_helper.py

在目錄/SlowFast-master/slowfast/datasets/下檔案：ava_helper.py，如下圖：

只需要一個修改：

#AVA_VALID_FRAMES = range(902, 1799)
AVA_VALID_FRAMES = range(1, 6)

把原來的 range(902, 1799) 改為 range(1, 6) 這是因為圖片的編號改為了1~5.

4.4 運行

python tools/run_net.py --cfg demo/AVA/SLOWFAST_32x2_R101_50_50s2.yaml

檢測結果如下：

視頻已經上傳到了B站：B站視頻