Computer vision · Engineering prototype · August 2025
Object Detection in Aerial Images
Building a custom anchor-based PyTorch detector for aerial imagery, from polygon annotations and a ResNet-50 backbone to classification and localisation losses, evaluation code, and a Flask inference interface.
- Target classes
- 6
- Labelled images
- 203
- Annotated objects
- 799
The problem
Locate and classify cars, houses, roads, swimming pools, trees, and yards in aerial images. Objects differ in scale and shape, and the supplied labels describe polygons. The project explores how to turn those annotations into axis-aligned detection targets and connect a feature extractor, custom anchors, and prediction heads into a trainable detector.
My contribution
- Implemented a dataset loader that reads normalised polygon coordinates, converts them to pixel coordinates, and derives bounding boxes from their minimum and maximum extents.
- Built a detector from a pretrained ResNet-50 feature extractor, a custom anchor generator, and separate classification and bounding-box regression heads.
- Implemented IoU-based anchor matching, box-offset encoding, and a combined cross-entropy and Smooth L1 training objective, with checkpoint saving and device selection.
- Added evaluation utilities and a prototype Flask POST /predict interface for loading an image and returning class, confidence, and bounding-box fields.
Approach
Prepare polygon annotations
Use the supplied training, validation, and test image/label pairs. Convert each polygon to an axis-aligned box, preserve its class ID, and resize images using an aspect-ratio-preserving size limit before tensor conversion.
Extract features and generate anchors
Use ImageNet-pretrained ResNet-50 features with configurable layer freezing. Anchor sizes of 32, 64, and 128 pixels combine with aspect ratios of 0.5, 1, and 2 to produce nine reference boxes per feature-map location.
Predict and optimise
Separate convolutional heads output class logits and four box offsets per anchor. Training matches anchors by IoU, ignores an intermediate overlap band, and combines classification loss with Smooth L1 localisation loss. The script is configured for Adam at a learning rate of 0.0001.
Evaluate and expose an interface
The evaluation path decodes offsets, filters confidence scores, applies non-maximum suppression, and calculates detection metrics. A separate Flask service path provides an inference interface; consistency between these paths remains part of the prototype's validation work.
Implementation & dataset
The prototype integrates data preparation, feature extraction, anchor generation, training, and an inference interface. Evaluation and post-processing need further validation.
| Partition | Images | Annotation files | Labelled objects |
|---|---|---|---|
| Training | 142 | 142 | 565 |
| Validation | 41 | 41 | 144 |
| Test | 20 | 20 | 90 |
Each image has a corresponding polygon-label file. Counts cover the supplied training/validation/test dataset; the separate high-resolution drone-image folder is outside these partitions.
- Polygon-to-box conversion makes the annotation format usable by an anchor-based detector, while simplifying the original object outlines to axis-aligned rectangles.
- The dataset contains 230 car annotations and 194 tree annotations, compared with 44 swimming-pool and 62 yard annotations. Per-class evaluation is needed alongside any aggregate score.
- The current training script selects its saved best checkpoint by training loss because evaluation metrics were unresolved. A lower training loss alone does not establish better detection quality.
Object-class distribution
| Class | Training | Validation | Test | Total |
|---|---|---|---|---|
| car | 169 | 41 | 20 | 230 |
| house | 131 | 40 | 26 | 197 |
| road | 50 | 15 | 7 | 72 |
| swimming pool | 34 | 6 | 4 | 44 |
| tree | 132 | 34 | 28 | 194 |
| yard | 49 | 8 | 5 | 62 |
Counts refer to annotated objects, so an image can contribute several objects and classes.
Experiments & analysis
Select a figure to view it at full size.
Inspect the detector composition
The detector connects its feature extractor, anchor generator, and prediction heads. Its forward pass returns class logits, box offsets, and anchors for loss calculation or decoding.
Python / PyTorch · Source excerpt
from torch import nn
from models.detection_head import DetectionHead
from models.feature_extractor import FeatureExtractor
from models.anchor_generator import AnchorGenerator
class ObjectDetector(nn.Module):
def __init__(self,
num_classes=6,
anchor_sizes=(32, 64, 128),
aspect_ratios=(0.5, 1.0, 2.0),
stride=32):
super(ObjectDetector, self).__init__()
self.feature_extractor = FeatureExtractor(pretrained=True, trainable_layers=3)
self.anchor_generator = AnchorGenerator(anchor_sizes, aspect_ratios, stride=stride)
self.detection_head = DetectionHead(
in_channels=2048,
num_classes=num_classes,
num_anchors=len(anchor_sizes)*len(aspect_ratios)
)
def forward(self, images):
device = images.device
features = self.feature_extractor(images)
anchors = self.anchor_generator(features, device=device)
class_logits, bbox_reg = self.detection_head(features)
return {
'class_logits': class_logits,
'bbox_reg': bbox_reg,
'anchors': anchors
}Limitations & what I learned
- Detection performance requires further development and validation before deployment.
- Foreground/background label handling and box decoding need to be consistent across training, evaluation, and the API. The current source has mismatches in those conventions, and the service path does not apply the evaluation path's non-maximum suppression.
- The custom AP calculation pools detections across classes. A validated per-class AP implementation and class averaging are needed before reporting standard mAP figures.
Where I would take this next
- Align foreground IDs and background logits, and add tests for anchor/box encoding and decoding before interpreting metric results.
- Use shared post-processing for evaluation and the service, including confidence filtering, clipping, and non-maximum suppression.
- Validate per-class AP with a standard evaluator, inspect predicted boxes against annotations, and select checkpoints using validated detection metrics.
- Evaluate augmentation, anchor coverage, and a reference detector once the correctness checks pass.