Speaker
Description
Abstract
Thermal imaging presents challenges for image registration due to uncorrelated temperature variations among objects, low inherent texture, reduced feature saliency, and modality-dependent contrast behaviour that differs from visible-spectrum imagery. These issues are particularly pronounced in thermal-to-thermal and thermal-to-visible registration tasks, where traditional handcrafted feature detectors and descriptors often exhibit poor repeatability and limited cross-modality robustness. As a result, learning-based approaches have become increasingly prominent in infrared thermography workflows. This work investigates deep point descriptors for thermal image registration under two scenarios: (i) mono-modality registration between thermal images, and (ii) cross-modality registration between thermal and visible (RGB) images. While classic handcrafted approaches (e.g., SIFT, ORB, and variants) remain widely used for visible-spectrum imaging, their performance often degrades on thermal data due to low feature saliency and poor repeatability across modalities. Recent literature indicates that machine learning (ML) and deep learning (DL) based methods consistently outperform classical pipelines by learning modality-invariant and thermally stable feature representations. Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) have emerged as dominant backbone architectures for learning robust thermal descriptors, with CNNs providing strong locality and ViTs offering improved global context aggregation.
Motivated by this, we present a comparative analysis of CNN- and ViT-based point descriptors for thermal registration. Models were trained and evaluated using publicly available datasets released under a CC-BY-4.0 license. Two different test sets were prepared to evaluate descriptor performance using positive (correct matches) and negative (incorrect matches) pairs. Metrics included mean and median L2 distances for positive and negative sample pairs, as well as False Positive Rate at 95% True Positive Rate (FPR@95TPR), a standard accuracy metric in feature matching.
In a subset of our evaluation, Test Set 1 (167,768 matching pairs) showed that both backbones achieved 95% TPR, with CNN marginally outperforming ViT on positive pair discrimination: the ViT positive mean was 0.2923 vs. 0.2045 for CNN, and the positive median was 0.2703 vs. 0.2014. Negative mean distances remained similarly high for both models (1.3977 for CNN vs. 1.3915 for ViT), suggesting comparable separation for non-matching descriptors. CNN achieved a slightly lower FPR@95TPR (0.00528 vs. 0.00552), reflecting marginally better suppression of false matches at the same recall level. Similar trends were observed in Test Set 2 (27,636 pairs), where CNN achieved lower positive mean (0.2035 vs. 0.3121) and median (0.2008 vs. 0.2901 ) and with ViT having modestly higher FPR@95TPR (4.37e-05 vs. 1.29e-05). While negative means remained high for both (1.3909 for CNN vs. 1.3971 for ViT). In both cases, CNN models demonstrated improved feature discrimination and retained slightly tighter control over false positives
