Federated Learning for Multi-Modal Pulmonary Disease Diagnosis in Real-Time Clinical Environments

About the project

About the Project Pulmonary diseases such as chronic obstructive pulmonary disease (COPD), pneumonia, and lung cancer are global health challenges. computed tomography (CT) and Magnetic resonance imaging (MRI) imaging are vital for diagnosis, but single-modality limitations reduce accuracy. Incorporating multi-modal imaging (CT, MRI) with clinical data could enhance diagnostic precision. This project’s idea centres around designing a multi-modal neural network model that combines CT and MRI imaging with patient metadata (e.g., age, smoking history, lab results) in a privacy-preserving, real-time system [1]. Using Federated Learning (FL) where multiple entities (often referred to as clients) collaboratively train a model while keeping their data decentralized; each hospital or clinical site trains the model on its local dataset and contributes encrypted updates to a central model without sharing sensitive patient data. State-of-the-art model interpretability methods (e.g., cross-modal attention) will explain the model's decision-making for physicians. As most existing FL studies focus on taking unimodal data, such as image and text, as the model input and resolving the heterogeneity challenge, i.e., the challenge of non-identical distribution (non-IID) caused by a data distribution imbalance related to data labels and data amount. This project will address the above-mentioned research problems in addition of integration of different data modalities, scaling of the FL models, and additional layers of data security while training medical data. The ideal candidate must demonstrate knowledge of deep learning, federated learning, and medical imaging.

Research areas