🕹️ [TIP'25] UFPF: A Universal Feature Perception Framework for Microscopic Hyperspectral Medical Images

💫 Abstract :

In recent years, deep learning has shown potential in medical microscopic hyperspectral imaging. Still, current research is mostly limited to a single task and lacks systematic integration of spatial-spectral features and optimization of model perception techniques to fully explore the clinical value of hyperspectral data.

🏇🏄🏂🚵🎳🏊🚴 Point done ：

In this paper, we propose a microscopic hyperspectral universal feature perception framework (UFPF) that integrates convolutional neural network (CNN), Transformers, and Mamba structures to extract spatial-spectral features of hyperspectral data comprehensively. This innovative framework captures different sequential spatial nearest-neighbor relationships through a hierarchical corner-to-center mamba structure, ensuring that key spatial dependencies and contextual information are not overlooked. On this basis, a dual-path spatial-spectral joint perception module is developed to improve model accuracy, which processes spatial and spectral information in parallel. In addition, the mamba-transformer mix-encoder is designed to enhance global understanding and spatial interaction capture capability. The experimental results on multiple datasets have shown that this framework significantly improves classification and segmentation performance, supporting the clinical application of medical hyperspectral data.

Acknowledgements

We highly appreciate all the dataset owners for providing the public dataset to the community.

Reference：

@ARTICLE{11114798,
  author={Qin, Geng and Liu, Huan and Li, Wei and Zhang, Xueyu and Guo, Yuxing and Xia, Xiang-Gen},
  journal={IEEE Transactions on Image Processing}, 
  title={UFPF: A Universal Feature Perception Framework for Microscopic Hyperspectral Images}, 
  year={2025},
  volume={34},
  number={},
  pages={5513-5526},
  keywords={Hyperspectral imaging;Feature extraction;Transformers;Microscopy;Image segmentation;Data models;Computer architecture;Adaptation models;Deep learning;Accuracy;Microscopic hyperspectral images;multi-task;mamba models},
  doi={10.1109/TIP.2025.3594151}}