Learning Prognostic World Models from Longitudinal Medical Data (Botnar-2026-07)

About the project

About the Project Background and Rationale Longitudinal medical imaging provides a rich but underused account of disease evolution. Clinicians interpret serial scans together with reports and clinical metadata to determine whether pathology is stable, progressing, or responding to treatment, yet most medical AI systems remain static: they classify a single scan, predict a downstream endpoint, or generate plausible images without explicitly representing how patient state changes over time [1-4]. This is a clinically important limitation because decisions about monitoring, intervention, and risk stratification depend on temporal structure rather than cross-sectional pattern recognition alone [3,5]. World models offer a natural framework by learning a latent state and a transition function that predicts future states, optionally conditioned on actions [12,13]. Digital-twin work in osteoarthritis and early medical world models for tumour evolution and surgical scenes demonstrate feasibility [5,8,9], but the field still lacks latent states explicitly optimised for prognosis. Generative fidelity alone does not guarantee prognostic validity [2,10], and models may exploit scanner, site, or cohort artifacts rather than disease-relevant structure [11]. Research Aim and Questions The thesis aims to develop a multimodal predictive world model that learns patient-specific disease dynamics from serial imaging, reports, and structured clinical data; supports intervention-aware simulation; and produces clinically useful forecasts with calibrated uncertainty. It will: (1) test whether latent dynamics improve multi-step forecasting, calibration, and time-to-event prediction over static, survival, and non-generative deep baselines [3,7]; (2) whether multimodal inputs yield more personalised and robust representations than imaging alone [2,10]; (3) whether action-conditioned transitions generate plausible divergent futures under treatments or trajectory-altering events [8,9]; (4) whether scenario-based forecasts improve interpretation compared with a single risk estimate; and (5) whether robustness-constrained latent states transfer better across datasets, institutions, scanners, and subgroups. Potential Methodological Framework The model will combine four coupled elements. A multimodal encoder will map imaging, reports, and structured metadata at time ‘t’ to a latent disease state ‘zt’. A JEPA-inspired temporal predictor will estimate future representations ‘z(t+delta)’, encouraging the representation to capture predictable, prognostically meaningful structure rather than reconstructing every visual detail [14,15]. An action-conditioned dynamics module will learn transitions from ‘(zt, at)’ to ‘z(t+1)’, where actions may be recorded treatments, proxy changes, or inferred latent actions [8,9]. A conditional VAE or latent-diffusion decoder will render predicted states as simulated future images for interpretation, but will not define the representation. Training will jointly optimise latent prediction, multimodal alignment, prognostic supervision, uncertainty-aware endpoint prediction, and invariant or adversarial regularisation against nuisance variation [16]. Study Design and Evaluation The study will use retrospective longitudinal cohorts with patient-level temporal splits and external validation where feasible. Eligible datasets must provide repeated imaging, visit-aligned clinical data, a progression or event endpoint, and acquisition metadata for leakage prevention and shift analysis. The Osteoarthritis Initiative is a strong initial case study because it provides serial MRI and radiographs with rich outcomes [6], followed by transfer to suitable neuroimaging or oncology cohorts. Evaluation will cover multi-step latent error, biomarker trajectories, AUROC/AUPRC, concordance, and calibration; feature-space fidelity and blinded expert review of simulated futures; observed-treatment and quasi-experimental plausibility checks; and robustness under site, scanner, and subgroup shifts. Stochastic latent rollouts combined with ensembles or Monte Carlo dropout will produce risk bands, alternative trajectories, and image uncertainty maps. Expected Contributions and Responsible AI The expected contributions are a unified JEPA-based, intervention-aware, robustness-constrained world model for longitudinal medicine; empirical evidence comparing predictive and generative objectives and testing multimodal benefit and cross-setting transfer; and patient-specific scenario forecasts that support monitoring, trial design, and shared decision-making. Responsible translation will require subgroup evaluation, calibrated uncertainty, transparent reporting, explicit labelling of all simulated images, and human decision support rather than autonomous diagnosis. Code will be released openly, with dataset datasheets where source data cannot be shared. Supervisors: Paula Dhiman (primary supervisor), NDORMS, CSM, University of Oxford. paula.dhiman@csm.ox.ac.uk Rafael Pinedo-Villanueva, NDORMS, University of Oxford. rafael.pinedo@ndorms.ox.ac.uk Training The Botnar Research Centre plays host to the University of Oxford's Institute of Musculoskeletal Sciences and the Centre for Statistics in Medicine (CSM), which enables and encourages research and education into the causes of musculoskeletal disease and their treatment and medical statistics. Training will be provided in techniques including medical statistics, prediction model research, simulations studies and clinical data management. A core curriculum of lectures will be taken in the first term to provide a solid foundation in a broad range of subjects including musculoskeletal biology, inflammation, epigenetics, translational immunology, data analysis and the microbiome. Students will also be required to attend regular seminars within the Department and those relevant in the wider University. Students will be expected to present data regularly in Departmental and CSM seminars, and to attend external conferences to present their research globally, with limited financial support from the Department. Students will also have the opportunity to work closely with the Oxford Psoriatic Arthritis Centre (Ox-PACE, NDORMS) and the Biostatistics, Evidence Synthesis, Test Evaluation and prediction Models (BESTEAM, Applied Health, University of Birmingham). Students will have access to various courses run by the Medical Sciences Division Skills Training Team and other Departments. All students are required to attend a 2-day Statistical and Experimental Design course at NDORMS (information will be provided once accepted to the programme). How to Apply Please contact the relevant supervisor(s), to register your interest in the project, and, if required, the departmental Education Team ( graduate.studies@ndorms.ox.ac.uk ), who will be able to advise you of the essential requirements for the programme and provide further information on how to make an official application. Interested applicants should have, or expect to obtain, a first or upper second-class BSc degree or equivalent in a relevant subject and will also need to provide evidence of English language competence (where applicable). The application guide and form is found online and the DPhil will commence in October 2027. Applications should be made to one of the following programmes using the specified course code. D.Phil in Clinical Epidemiology and Medical Statistics (course code: RDNNRA1) For further information, please visit http://www.ox.ac.uk/admissions/graduate/applying-to-oxford . Applications open mid-September Application deadline: 12:00 on 1st December

Research areas