計算機視覺可解釋性——卷積神經網路中間層的可視化-有解無憂

這段代碼的步驟大致如下：

1.處理單張圖作為網路的輸入，
2.根據給定的layer層，獲取該層輸出結果features，
3.考慮到features的形狀為[batch_size, filter_nums, H, W],提取其中第一個過濾器得到的結果的feature，
4.以一張圖作為輸入的情況下，我們得到的feature即為[H,W]大小的tensor
5.將tensor轉為numpy，然后歸一化[0,1]，最后乘以255，使得范圍為[0,255]，
6.得到灰度影像保存，

————————————————————————————————

import cv2
import numpy as np
import torch
from torch.autograd import Variable
from torchvision import models


def preprocess_image(cv2im, resize_im=True):
    """
    Processes image for CNNs
    Args:
    PIL_img(PIL_img):Image to process
    resize_im(bool):Resize to 224 or not
    :return:
    im_as_var(Pytorch variable):Variable that contains processed float tensor
    """
    # mean and std list for channels (ImageNet)
    mean = [0.485, 0.456, 0.406]
    std = [0.229, 0.224, 0.225]
    # Resize image
    if resize_im:
        cv2im = cv2.resize(cv2im, (224, 224))
    im_as_arr = np.float32(cv2im)
    im_as_arr = np.ascontiguousarray(im_as_arr[..., ::-1])
    im_as_arr = im_as_arr.transpose(2, 0, 1)  # Convert array to D,W,H
    # Normalize the channels
    for channel, _ in enumerate(im_as_arr):
        im_as_arr[channel] /= 255
        im_as_arr[channel] -= mean[channel]
        im_as_arr[channel] /= std[channel]
    # Convert to float tensor
    im_as_ten = torch.from_numpy(im_as_arr).float()
    # Add one more channel to the beginning. Tensor shape =1,3,224,224
    im_as_ten.unsqueeze(0)
    # Convert to Pytorch variable
    im_as_var = Variable(im_as_ten, requires_grad=True)
    return im_as_var


class FeatureVisualization():
    def __init__(self, img_path, selected_layer):
        self.img_path = img_path
        self.selected_layer = selected_layer
        self.pretrained_model = models.vgg16(pretrained=True).features

    def process_image(self):
        img = cv2.imread(self.img_path)
        img = preprocess_image(img)
        return img

    def get_feature(self):
        # input = Variable(torch.randn(1, 3, 224, 224))
        input = self.process_image()
        x = input
        x = x.unsqueeze(0)
        for index, layer in enumerate(self.pretrained_model):
            x = layer(x)
            if (index == self.selected_layer):
                return x

    def get_single_feature(self):
        features = self.get_feature()
        # print("get_single_features: ", features.shape)
        feature = features[:, 0, :, :]
        # print("get_single_feature: ", feature.shape)
        feature = feature.view(feature.shape[1], feature.shape[2])
        return feature

    def save_feature_to_img(self, i):
        # to numpy
        feature = self.get_single_feature()
        feature = feature.data.numpy()
        # use sigmod to [0,1]
        feature = 1.0 / (1 + np.exp(-1 * feature))
        # to [0,255]
        feature = np.round(feature * 255)
        print("ok feature.shape: ", feature.shape)
        cv2.imwrite(str(i) + ".jpg", feature)


if __name__ == "__main__":
    for i in range(30):
        myClass = FeatureVisualization("31.jpg", i)
        # print(myClass.pretrained_model)
        myClass.save_feature_to_img(i)

原圖如下

在這里插入圖片描述
—————————————————————————————————

可視化結果如下

第1層
在這里插入圖片描述
—————————————————————————————————
第2層

—————————————————————————————————
第3層

—————————————————————————————————
第4層
在這里插入圖片描述 —————————————————————————————————
第5層
—————————————————————————————————
第6層

—————————————————————————————————
第7層
在這里插入圖片描述 —————————————————————————————————
第8層
—————————————————————————————————
第9層
—————————————————————————————————
第10層
在這里插入圖片描述 —————————————————————————————————
第11層
—————————————————————————————————
第12層
—————————————————————————————————
第13層
在這里插入圖片描述 —————————————————————————————————
第14層
—————————————————————————————————
第15層

—————————————————————————————————
第16層
在這里插入圖片描述 —————————————————————————————————
第17層

—————————————————————————————————
第18層

—————————————————————————————————
第19層
在這里插入圖片描述 —————————————————————————————————
第20層
___________________________________________________________
第21層
___________________________________________________________