《智能計算系統》實驗-7-1-YOLOv3

在做《智能計算系統》綜合實驗7-1-YOLOv3時，遇到了很多問題，實驗書程序不全，現將整個實驗流程梳理如下，以對其他讀者有所裨益：

一、搭建環境

新建容器v7（非v7-update1）

二、nms_detection.h實作

1. 補全nms_detection.h，實作函式:

__mlu_func__ void nms_detection(
        int& output_box_num,
        NMS_DT* output_data,
        Addr dst,
        NMS_DT* input_data_score,
        NMS_DT* input_data_box,
        Addr src,
        NMS_DT *buffer,
        int buffer_size,
        NMS_DT* sram,
        SplitMode split_mode,
        int input_box_num,
        int input_stride,
        int output_stride,
        int keepNum,
        NMS_DT thresh_iou,
        NMS_DT thresh_score,
        int save_method){...}

只需考慮以下情況：

src == NRAM
split_mode == NMS_BLOCK
save_method == 1
MODE == 1

2. 將/opt/code_chap_7_student/yolov3/bangc/PluginYolov3DetectionOutputOp復制到/opt/code_chap_7_student/env/Cambricon-CNPlugin-MLU270/pluginops路徑下

3. 初始化環境
每次進入系統，都需進入env目錄, 執行 source env.sh命令

cd /opt/code_chap_7_student/env
source env.sh  

4. 編譯

cd /opt/code_chap_7_student/env/Cambricon-CNPlugin-MLU270
./build_cnplugin.sh  

5. 將./build/libcnplugin.so復制到../neuware/lib64

cp ./build/libcnplugin.so /opt/code_chap_7_student/env/neuware/lib64

6. 將./pluginops/PluginYolov3DetectionOutputOp/cnplugin.h復制到./neuware/include

cp ./pluginops/PluginYolov3DetectionOutputOp/cnplugin.h /opt/code_chap_7_student/env/neuware/include

三、算子集成

補全/opt/code_chap_7_student/yolov3/tf-implementation/tf-1.14-detectionoutput目錄下檔案，

1. MLULib封裝

/*
    mlu_lib_ops.h //line 924
    mlu_lib_ops.cc //line 1918
*/
tensorflow::Status CreateYolov3DetectionOutputOp(
   MLUBaseOp** op, 
   MLUTensor** input_tensors, 
   MLUTensor** output_tensors,
   cnmlPluginYolov3DetectionOutputOpParam_t param){...}
   
 tensorflow::Status ComputeYolov3DetectionOutputOp(
   MLUBaseOp* op,
   MLUCnrtQueue* queue,
   void* inputs[], 
   int input_num,
   void* outputs[],
   int output_num){...}

2. MLUOp封裝

/*
    mlu_ops.h //line 530
*/
struct MLUYolov3DetectionOutputOpParam{};
DECLARE_OP_CLASS(MLUYolov3DetectionOutput);

/*
    yolov3detectionoutput.cc //line 12
*/
Status MLUYolov3DetectionOutput::CreateMLUOp(std::vector<MLUTensor*> &inputs, std::vector<MLUTensor*> &outputs, void *param){...}

Status MLUYolov3DetectionOutput::Compute(const std::vector<void *> &inputs,
 const std::vector<void *> &outputs, cnrtQueue_t queue){...}

3. MLUStream封裝

/*
    mlu_stream.h //line 141
*/
Status Yolov3DetectionOutput(
     OpKernelContext* ctx,
     Tensor* tensor_input0,
     Tensor* tensor_input1,
     Tensor* tensor_input2,
     int batchNum,
     int inputNum,
     int classNum,
     int maskGroupNum,
     int maxBoxNum,
     int netw,
     int neth,
     float confidence_thresh,
     float nms_thresh,
     int* inputWs,
     int* inputHs,
     float* biases,
     Tensor* output1,
     Tensor* output2){...}

4. MLUOpKernel封裝

/*
    yolov3_detection_output_op_mlu.h //line 49
*/
void ComputeOnMLU(OpKernelContext* context) override{...}

/*
    yolov3_detection_output_op.cc //line 23
*/
namespace tensorflow{...}

5. 算子注冊

/*
    image_ops.cc //line 1007
*/
REGISTER_OP("Yolov3DetectionOutput"){...}

四、框架編譯

1. BUILD修改(已完成)

2. 將/opt/code_chap_7_student/yolov3/tf-implementation/tf-1.14-detectionoutput下各檔案依次放入對應檔案夾，可利用cp命令：

cp ./tf-implementation/tf-1.14-detectionoutput/BUILD ../env/tensorflow-v1.10/tensorflow/core/kernels/BUILD
cp ./tf-implementation/tf-1.14-detectionoutput/image_ops.cc ../env/tensorflow-v1.10/tensorflow/core/ops/image_ops.cc
cp ./tf-implementation/tf-1.14-detectionoutput/yolov3_detection_output_op.cc ../env/tensorflow-v1.10/tensorflow/core/kernels/yolov3_detection_output_op.cc
cp ./tf-implementation/tf-1.14-detectionoutput/yolov3_detection_output_op_mlu.h ../env/tensorflow-v1.10/tensorflow/core/kernels/yolov3_detection_output_op_mlu.h
cp ./tf-implementation/tf-1.14-detectionoutput/mlu_lib_ops.cc ../env/tensorflow-v1.10/tensorflow/stream_executor/mlu/mlu_api/lib_ops/mlu_lib_ops.cc
cp ./tf-implementation/tf-1.14-detectionoutput/mlu_lib_ops.h ../env/tensorflow-v1.10/tensorflow/stream_executor/mlu/mlu_api/lib_ops/mlu_lib_ops.h
cp ./tf-implementation/tf-1.14-detectionoutput/mlu_ops.h ../env/tensorflow-v1.10/tensorflow/stream_executor/mlu/mlu_api/ops/mlu_ops.h
cp ./tf-implementation/tf-1.14-detectionoutput/yolov3detectionoutput.cc ../env/tensorflow-v1.10/tensorflow/stream_executor/mlu/mlu_api/ops/yolov3detectionoutput.cc
cp ./tf-implementation/tf-1.14-detectionoutput/mlu_stream.h ../env/tensorflow-v1.10/tensorflow/stream_executor/mlu/mlu_stream.h

3. 框架編譯

rm -rf /root/.cache/bazel/_bazel_root
cd /opt/code_chap_7_student/env/tensorflow-v1.10
./build_tensorflow-v1.10_mlu.sh

在編譯時，要先洗掉/root/.cache/bazel/_bazel_root檔案夾，若報錯，則洗掉/root/.cache/bazel/_bazel_root/*重試

五、在線推理

1. pb->pbtxt

1）將/opt/Cambricon-Test/models/yolov3/目錄下yolov3_int8_bang_shape_new.pb復制到/opt/code_chap_7_student/yolov3/yolov3-bcl/demo目錄

cd /opt/code_chap_7_student/yolov3/yolov3-bcl/demo
cp /opt/Cambricon-Test/models/yolov3/yolov3_int8_bang_shape_new.pb ./

2）將.pb轉為.pbtxt

python /opt/code_chap_7_student/tools/pb_to_pbtxt/pb_to_pbtxt.py yolov3_int8_bang_shape_new.pb yolov3_int8_bang_shape_new.pbtxt

2. 修改yolov3_int8_bang_shape_new.pbtxt，添加node{...}

執行該程序時，由于.pbtxt檔案過大，打開檔案并修改導致連接斷開，可利用shell命令添加相關內容，例如，將node{...}存放在pb_node_append.txt檔案中（包含library{...}），類似：

//pb_node_append.txt
node {...}
library {...}

執行

sed -i "/^library/,/^38\n}$/d" yolov3_int8_bang_shape_new.pbtxt
cat ./pb_node_append.txt >> yolov3_int8_bang_shape_new.pbtxt

3. pbtxt->pb

python /opt/code_chap_7_student/tools/pbtxt_to_pb/pbtxt_to_pb.py ./yolov3_int8_bang_shape_new.pbtxt yolov3_int8.pb

4. 修改./run_evaluate.sh中MODEL_PATH="./yolov3_int8.pb"

5. 運行./run_aicse.sh

六、可能遇到的問題

1. 框架編譯時執行./build_tensorflow-v1.10_mlu.sh，fetching不通過，洗掉/root/.cache/bazel/_bazel_root/*重試

2. ***cannot find ，需要初始化環境變數，即在./env目錄下執行source env.sh命令，需要注意的是，每一次登錄后進入開發容器均需要執行source env.sh命令

3. 框架編譯時類似如下錯誤

Executing genrule //tensorflow/python/keras/api:keras_python_api_gen_compat_v2 failed (Exit 1): bash failed: error executing command

/opt/code_chap_7_student/env/tensorflow-v1.10/tensorflow/python/keras/api/BUILD:28:1: Executing genrule //tensorflow/python/keras/api:keras_python_api_gen_compat_v1 failed (Exit 1): bash failed: error executing command 

需要確定是否在對CNPlugin編譯后，將./build/libcnplugin.so復制到./neuware/lib64，

七、其他

1. 開發手冊下載：檔案中心 – 寒武紀開發者社區

2. BUILD檔案語法：bazel C++語法入門 - 簡書

3. PB檔案格式：Tensorflow模型持久化與恢復_jinying2224的博客-CSDN博客