VSG-GAN: A high-fidelity image synthesis method with semantic manipulation in retinal fundus image

Abstract

In recent years, advancements in retinal image analysis, driven by machine learning and deep learning techniques, have enhanced disease detection and diagnosis through automated feature extraction. However, challenges persist, including limited dataset diversity due to privacy concerns and imbalanced sample pairs, hindering effective model training. To address these issues, we introduce the Vessel & Style Guided Generative Adversarial Network (VSG-GAN), an innovative algorithm building upon the foundational concept of GAN. In VSG-GAN, a generator and discriminator engage in an adversarial process to produce realistic retinal images. Our approach decouples retinal image generation into distinct modules: the vascular skeleton and background style. Leveraging style transformation and GAN inversion, our proposed Hierarchical Variational Autoencoder (HVAE) module generates retinal images with diverse morphological traits. Additionally, the Spatially-Adaptive De-normalization (SPADE) module ensures consistency between input and generated images. We evaluate our model on MESSIDOR and RITE datasets using various metrics, including Structural Similarity Index Measure (SSIM), Inception Score (IS), Fr´echet Inception Distance (FID), and Kernel Inception Distance (KID). Our results demonstrate the superiority of VSG-GAN, outperforming existing methods across all evaluation assessments. This underscores its effectiveness in addressing dataset limitations and imbalances. Our algorithm provides a novel solution to challenges in retinal image analysis by offering diverse and realistic retinal image generation. Implementing the VSG-GAN augmentation approach on downstream diabetic retinopathy classification tasks has shown enhanced disease diagnosis accuracy, further advancing the utility of machine learning in this domain.