Two Stage Detection

根据传统机器学习的流程，改变使用 CNN 提取 feature 进行目标检测

A. RCNN: NMS Series

第一步 input
第二步 提取 region proposals (大约有两千块) ，然后将所有的 proposal 统一大小。 使用 select search 算法提取 proposals(像素值相近与否区分物体)
第三步 使用 cnn 计算提取特征
第四步 分类检测 svm cls + nms + bbox regression

图片中有两个以上的相同物体使用 nms 算法可以找到两个，根据 iou 找极大值
Sort bbox according to score [here class score]
Get the 1st one. Remove those which has higher iou with it
Then repeat the last step until there’s no bbox

def nms(bounding_boxes, confidence_score, threshold):
    """
    describe： Non-max Suppression Algorithm
    :param bounding_boxes: list  Object candidate bounding boxes
    :param confidence_score: list  Confidence score of bounding boxes
    :param threshold: float IoU threshold

    :return picked_boxes: list Rest boxes after nms operation
    :return picked_score: list The score of rest boxes after nms operation
    """
    # If no bounding boxes, return empty list
    if len(bounding_boxes) == 0:
        return [], []

    # Bounding boxes
    boxes = np.array(bounding_boxes)

    # coordinates of bounding boxes
    start_x = boxes[:, 0]
    start_y = boxes[:, 1]
    end_x = boxes[:, 2]
    end_y = boxes[:, 3]

    # Confidence scores of bounding boxes
    score = np.array(confidence_score)

    # Picked bounding boxes
    picked_boxes = []
    picked_score = []

    # Compute areas of bounding boxes
    areas = (end_x - start_x + 1) * (end_y - start_y + 1)

    # Sort by confidence score of bounding boxes
    order = np.argsort(score)

    # Iterate bounding boxes
    while order.size > 0:
        # The index of largest confidence score
        index = order[-1]

        # Pick the bounding box with largest confidence score
        picked_boxes.append(bounding_boxes[index])
        picked_score.append(confidence_score[index])

        # Compute ordinates of intersection-over-union(IOU)
        x1 = np.maximum(start_x[index], start_x[order[:-1]])
        x2 = np.minimum(end_x[index], end_x[order[:-1]])
        y1 = np.maximum(start_y[index], start_y[order[:-1]])
        y2 = np.minimum(end_y[index], end_y[order[:-1]])

        # Compute areas of intersection-over-union
        w = np.maximum(0.0, x2 - x1 + 1)
        h = np.maximum(0.0, y2 - y1 + 1)
        intersection = w * h

        # Compute the ratio between intersection and union
        ratio = intersection / (areas[index] + areas[order[:-1]] - intersection)

        left = np.where(ratio < threshold)
        order = order[left]

    return picked_boxes, picked_score

softnms 解决相近物体的分类问题
高斯递减函数

B. Fast RCNN: ROI Series

改良 RCNN，第一步，整张图进行卷积，然后使用 roi project 的到 feature map。
第二步，roi pooling layer(取最大值) 调整到相同的大小，FC。其中整形化处理会损失很多像素值。
第三步，分类回归

roi align
precise roi pooling

C. Faster RCNN: RPN + Anchor

c1 backbone
THE AIM is Extract feature integratedly
输出：
B x C x H/16 x W/16
1 x 256 x 38 x 50
c2 rpn : region proposal network
使用 1x1 的 filter 进行卷积，一共为 18 个，生成的新的图片是 18-D 的:然后进行reshape，拿一张图片举个例子，图片的shape是（W，H，D=18），然后我们会把他reshape以进行softmax（进行softmax的matrix的一边需要等于 num of class，在这里是一个二分类，即是否含有物体，所以是2）。所以我们会把（W，H，D）reshape成（2，9WH）。这里很重要！！！！

❑ Represent an area in original image (a.k.a. where objects are)
❑ Have different scale & ratio to cover all kinds of objects
❑ 9 in total: 3 scales x 3 ratios
❑ 1 x 9 x 38 x 50 = 17100 anchors
❑ Coords of anchors are of original imgs

offset 回归找到 proposal

D. Some Resources

II. One Stage Detection

E. Yolo V1: 1st Trial

F. Yolo V2: Anchor

G. Yolo V3: FPN

H. Yolo V4: Tricks

I. RetinaNet: Focal Loss

J. Some Resources

K. Other method: SSD

III. Anchor Free Methods

L. Trend

M. CenterNet

N. FCos

IV. Some Code

O. Anchor / Prior Box

P. Evaluation

Q. NMS (Your Assignment)