Skip to main content

[Vision AI] Blackbox Lane Intrusion & Super-Resolution ANPR: 6-Stage End-to-End Pipeline & Real-Time Constraints (#1)

· 3 min read
Jihwan Park (jaybee)
Jihwan Park (jaybee)
AI & Robotics Engineer

6-Stage ANPR Pipeline Diagram

📌 Key Takeaways (TL;DR)

  • Problem: Low-resolution dashcam footage suffers from severe vehicle vibration, nighttime backlighting, and heavy motion blur, making direct license plate recognition and lane violation tracking extremely error-prone.
  • Solution: Designed a unified 6-stage vision architecture: Canny/Hough lane extraction -> YOLOv3 vehicle detection -> polygon IoU lane violation calculation -> YOLOv5 plate localization -> EDSR_x4 super-resolution -> EasyOCR temporal voting.
  • Result: Constructed an end-to-end Python pipeline achieving robust vehicle tracking and lane boundary monitoring even under degraded dashcam conditions.

1. Project Background: Why Dashcam Vision is Hard​

In intelligent transportation systems (ITS) and autonomous traffic enforcement, real-time lane intrusion detection and automatic number plate recognition (ANPR) are foundational capabilities. However, real-world dashcam footage presents severe optical challenges:

  1. Extreme Low Resolution & Blurred Textures: The license plate of a leading or overtaking vehicle often spans merely 30x15 to 50x25 pixels across a 640x480 frame, causing alphanumeric strokes to merge into indistinct blobs.
  2. Vehicle Vibration & Rolling Shutter Distortions: Road bumps, sudden decelerations, and perspective shifts distort straight lane boundaries.
  3. Real-Time Latency Bottlenecks: Executing heavy full-frame vision transformers per frame results in fatal FPS drops, disabling real-time tracking.

2. 6-Stage Hierarchical Pipeline Architecture​

To achieve both high precision and high throughput, we modularized the system into a 6-stage decoupled pipeline:

flowchart TD
    Raw["Raw Dashcam Frame 640x480"] --> L1["Stage 1: Canny + Hough Dynamic Lane Extraction"]
    Raw --> L2["Stage 2: YOLOv3-tiny Vehicle Detection"]
    L1 & L2 --> L3["Stage 3: Vehicle Footprint & Lane Polygon IoU Intrusion Metric"]
    L3 -- If Intrusion Triggered --> L4["Stage 4: Localized YOLOv5 Plate Detection in Vehicle ROI"]
    L4 --> L5["Stage 5: EDSR_x4 Deep 4x Super-Resolution"]
    L5 --> L6["Stage 6: EasyOCR + Regex + Temporal Majority Voting"]
    L6 --> Result["Confirmed Violation Plate: 64가1511"]

Final Detection and ANPR Recognition Output

Decoupled Processing Responsibilities​

  1. Lane Extraction: Trapezoidal dynamic ROI masking, Canny edge detection, and slope filtering isolate driving corridor bounds.
  2. Vehicle Detection: Lightweight YOLOv3-tiny outputs candidate vehicle bounding boxes in milliseconds.
  3. Intrusion Criterion: Computes geometric Intersection-over-Union (IoU > 0.05) between vehicle tire footprint and the lane boundary polygon.
  4. Plate Localization: YOLOv5 (lp_det.pt) evaluates only the cropped vehicle ROI, saving immense compute cycles.
  5. Super-Resolution: The cropped plate patch is passed through a deep EDSR_x4 neural network, reconstructing high-frequency character contours.
  6. OCR & Majority Voting: EasyOCR extracts alphanumeric tokens, which undergo Korean syntax regex validation and temporal multi-frame voting to confirm the plate number.

3. Core System Parameters (CONFIG)​

CONFIG = {
    "output_width": 640,
    "output_height": 480,
    "conf_threshold_vehicle": 0.5,
    "nms_threshold_vehicle": 0.4,
    "min_vehicle_w": 70,
    "min_vehicle_h": 70,
    "lane_iou_threshold": 0.05,
    "detection_interval": 15,
    "min_plate_area_for_ocr": 1200,
    "plate_history_size": 7,
    "plate_confirm_threshold": 3,
    "sr_model": "EDSR_x4.pb",
    "yolo_plate_model": "lp_det.pt"
}

4. Looking Forward​

In Post #2, we dive deep into Canny edge filtering, probabilistic Hough transform mathematics (HoughLinesP), slope-based outlier rejection, and the geometric polygon IoU intrusion criterion.