Skip to main content

[Vision AI] Hierarchical Object Detection: YOLOv3 Vehicle ROI, YOLOv5 Plate Detector & CSRT Tracking (#3)

· 2 min read
Jihwan Park (jaybee)
Jihwan Park (jaybee)
AI & Robotics Engineer

YOLOv5 Plate Detection Localization

📌 Key Takeaways (TL;DR)

  • Problem: Searching for tiny plates (30x15 pixels) across the raw 640x480 frame generates massive false positives from roadside signs and drops inference speed below 10 FPS.
  • Solution: Designed a two-stage hierarchical architecture: broad vehicle detection (YOLOv3-tiny) crops the vehicle ROI, within which a fine-tuned YOLOv5 (lp_det.pt, conf=0.6) locates the license plate. Integrated OpenCV CSRT correlation filter tracking with a 15-frame redetection interval.
  • Result: Eliminated 98% of background false positives and slashed GPU overhead by 80%, sustaining real-time 50+ FPS continuous tracking.

1. The Pitfalls of Global Detection & The Hierarchical Solution​

Running plate detectors across the full 640x480 image causes two critical flaws:

  1. Background Clutter: Street advertisements, road billboards, and store signs trigger high false-alarm rates.
  2. Resolution Downsampling Loss: Resizing 640x480 to typical model inputs (e.g., 640x640) squashes tiny 30-pixel plates into sub-feature extinction.

Our two-stage hierarchical pipeline addresses this by constraining the search space:

vehicle_boxes = detect_vehicles_yolov3(frame, net=vehicle_net, conf_thresh=0.5)

for box in vehicle_boxes:
    x1, y1, x2, y2 = box
    vehicle_roi = frame[y1:y2, x1:x2]
    plate_boxes = plate_model(vehicle_roi)
    for pbox in plate_boxes:
        px1, py1, px2, py2 = pbox
        global_plate_box = [x1 + px1, y1 + py1, x1 + px2, y1 + py2]

2. Real-Time Latency Mitigation: CSRT Tracker Integration​

Continuous Tracking with OpenCV CSRT

if frame_count % CONFIG["detection_interval"] == 0:
    boxes = detect_vehicles_yolov3(frame)
    tracker = cv2.TrackerCSRT_create()
    tracker.init(frame, tuple(boxes[0]))
else:
    success, tracked_box = tracker.update(frame)
    if not success:
        tracker = None

3. Looking Forward​

In Post #4, we explore deep EDSR_x4 super-resolution to reconstruct crisp alphanumeric strokes from blurry low-resolution plate crops, combined with multi-frame image stacking.