Trustworthy Agentic AI for Clinical Documentation: Safe, Transparent and Human-Centred Electronic Health Records

About the project

About the Project Generative AI is rapidly transforming healthcare documentation. Large language models (LLMs) are increasingly being integrated into electronic health record (EHR) systems to draft clinical notes, summarise patient histories and support clinical coding. The next generation of these technologies is evolving beyond passive "AI scribes" towards agentic systems that can actively coordinate, assemble and refine documentation across multiple information sources. While these advances offer significant opportunities to reduce administrative burden and improve clinical efficiency, they also introduce important safety and governance challenges. As AI-generated and clinician-authored content become increasingly interwoven within the health record, it becomes more difficult to determine where information originated, verify the evidence supporting individual statements, or detect fabricated ("hallucinated") or misleading content before it influences patient care. Developing AI systems that clinicians can understand, trust and safely oversee is therefore becoming one of the defining challenges in digital healthcare. This PhD will investigate how trustworthy agentic AI can be designed for clinical documentation, placing transparency, safety and meaningful human oversight at the centre of system design. Rather than treating safety as an afterthought, the research will develop architectures in which every AI-generated contribution is traceable to its underlying evidence, automatically validated where possible, and presented in ways that support informed clinical judgement. The project will draw upon structured and free-text EHR data while building on the supervisory team's expertise in health informatics, clinical natural language processing and trustworthy AI. Indicative research objectives include: Investigate the capabilities and limitations of agentic AI systems for clinical documentation tasks such as note generation, history summarisation and clinical coding, with particular emphasis on hallucinations, omissions, bias and unsafe reasoning. Design provenance and traceability frameworks that capture the origin, supporting evidence and confidence associated with AI-generated content, ensuring that human- and AI-authored contributions remain transparent throughout the lifecycle of the clinical record. Develop safety mechanisms, including retrieval-grounded generation, structured-data consistency checking, evidence verification and uncertainty estimation, to detect and intercept potentially unsafe outputs before they reach the clinician. Design and evaluate human-in-the-loop workflows that preserve meaningful clinical oversight while maximising usability, efficiency and trust. Evaluate the proposed architecture using a rigorous framework combining automated benchmarking with clinician-centred studies exploring trust, explainability, cognitive workload and error detection. The project sits at the intersection of health informatics, trustworthy AI, software engineering and human-centred system design. It addresses one of the most pressing challenges surrounding the safe adoption of generative AI in healthcare and offers opportunities to develop novel architectures, evaluation methodologies and governance approaches for AI-assisted clinical documentation. Expected research outputs include safety and provenance frameworks, agentic system architectures, evaluation benchmarks, software prototypes and high-quality publications in leading health informatics and artificial intelligence venues. Applicants should have a strong background in Computer Science, Artificial Intelligence, Software Engineering, Health Informatics or a closely related discipline, together with programming experience in Python. Experience with natural language processing, machine learning, LLMs or healthcare data would be advantageous but is not essential. Above all, we are looking for applicants with an interest in building trustworthy, dependable and human-centred AI systems for safety-critical environments. The project will make use of synthetic and appropriately governed clinical datasets where necessary, with opportunities to work with real-world healthcare data subject to ethical and information-governance approvals. The successful candidate will join a supportive, interdisciplinary and research-active supervisory team with expertise spanning health informatics, clinical AI and software engineering, and will be encouraged to publish in leading international venues while collaborating with healthcare and industry partners.

Research areas