YOLOv8 Object Detection & Multi-Object Tracking Optimization

Overview

During an Artificial Intelligence internship at SF Express, I customized the YOLOv8 object detection architecture and optimized it for real-world deployment, working on both accuracy improvements and inference efficiency.

Motivation

Off-the-shelf YOLOv8 needed to be adapted to a custom vision dataset and deployment environment. The goal was to systematically explore architectural and training changes rather than apply defaults blindly.

Approach

I integrated several attention and convolution modules — SE, CBAM, ECA, PSA, DCN, and dynamic convolution — into the C2f blocks of the YOLOv8 backbone, then ran controlled experiments across:

• Detection heads
• Label assignment strategies
• Loss functions
• Optimizers
• Learning rate schedulers

to understand performance and convergence trade-offs across configurations.

For tracking, I integrated ByteTrack for multi-object tracking on top of the detector and analyzed the resulting accuracy–latency trade-offs.

To make the model deployment-ready, I applied structured and unstructured pruning together with FP16 quantization. I also extended the pipeline to instance segmentation using YOLOv8n-seg, building a data augmentation pipeline with Albumentations and PyTorch's DataLoader (flipping, rotation, scaling, and other transforms) to improve generalization.

Results

CBAM attention gave the largest accuracy gain of the attention variants tested, at some latency cost from the DCN-based architectures. Pruning and FP16 quantization substantially reduced memory footprint and increased throughput with negligible accuracy loss (see the Results panel below for exact figures).

Challenges

Balancing accuracy gains from heavier attention/convolution modules (like DCN) against their inference-time cost was the central trade-off — the most accurate configuration was not the fastest one.

Lessons Learned

Controlled, isolated experiments (changing one architectural component at a time) were essential to attribute accuracy and latency changes correctly, rather than confounding multiple simultaneous changes.

Future Improvements

• Explore INT8 quantization on top of the existing FP16 pipeline
• Benchmark on additional edge hardware targets
• Expand the instance segmentation evaluation to a larger custom dataset

Availability

This project was developed as part of a corporate internship, so the source code and dataset are proprietary and not publicly available.