Improving Few-Shot-Segmentation of New Structures in Volumetric Medical Images by Support Set Optimization

Few-Shot Learning (FSL) offers a promising solution to the high annotation costs in medical image analysis by suggesting a solution for handling limited labeled data. In the typical FSL setup, a pre-trained model uses a small, annotated support set to segment a new, unlabeled query image. However, performance can be highly variable due to overfit, as the limited support set may not be representative of the query. We propose a novel method to improve FSL performance for image segmentation tasks by dynamically optimizing the support set based on representative features extracted from the query image. The query-aware choice of more representative support set exploits overfitting to effectively overfit to the query image and improve model performance without re-training or additional annotations. We validate our approach on the task of liver lesions detection and segmentation in contrast-enhanced abdominal CT scans (103 scans, 2,442 lesions). The method improved the F1 score by 8.5% (from 0.59 to 0.64) on a support set of 13 scans, with respect to simple support selection policies that do not consider the query. Our results demonstrate that query-aware support set optimization significantly enhances FSL performance for small structures.