Multimodal Large Language Model for Human-Centered Robots

About the project

About the Project Supervisory Team: Dr. Tan Viet Tuyen Nguyen This project explores the development of a Multimodal Large Language Model that empowers robots to understand and respond to humans through vision, language, and other sensory data. By enabling natural, adaptive, and context-aware communication, the research advances the next generation of intelligent, human-centered robotic systems. Robots are increasingly being integrated into various aspects of daily life, including education, healthcare, workplaces, and personal settings. These practical applications require that robots collaborate effectively with humans in shared environments to enhance human capabilities, prioritize human needs, and seamlessly fit into everyday contexts This project aims to develop a Multimodal Large Language Model framework that enables robots to interpret interaction contexts from multiple modalities, such as vision, language, audio, and other sensory data, and to engage users through diverse communication channels, including actions, speech, and visual outputs. The framework will allow robots to perceive multimodal signals collected from both the environment and human users during everyday interactions, build a deep contextual understanding, and respond appropriately across different modalities to achieve natural, adaptive, and human-centered interaction. You will have the opportunity to collaborate with other researchers in the following areas: Agents, Interaction, and Complexity research group Centre for Robotics Responsible AI project team You'll contribute to high-impact journals and conferences. You will have access to high-performance computers and a range of robot platforms to support your research. Entry requirements A UK 2:1 honours degree, or its international equivalent . You must have solid programming skills in Python and machine learning, and a strong interest in the intersection of machine learning and robotics. Prior experience with large language models or multimodal learning would be advantageous. Fees and funding UK, EU and Horizon Europe students are eligible for scholarships. CSC students are eligible for fee waivers. Funding for other international applicants is very limited and highly competitive. Overseas students who have secured or are seeking external funding are welcome to apply. How to apply Apply now programme type: research academic year: 2026/27 if you will be full time or part time faculty: Engineering and Physical Sciences search for programme: PhD Computer Science (7089) please add the name of the supervisor in section 2 of the application. Applications should include: research proposal your CV (resumé) 2 academic references degree transcripts/ certificates to date English language qualification (if applicable) Contact us Faculty of Engineering and Physical Sciences If you have a general question, email: feps-pgr-apply@soton.ac.uk Project leader For an initial conversation, email Dr. Tan Viet Tuyen Nguyen - tuyen.nguyen@soton.ac.uk The School of Electronics & Computer Science is committed to promoting equality, diversity inclusivity as demonstrated by our Athena SWAN award. We welcome all applicants regardless of their gender, ethnicity, disability, sexual orientation or age, and will give full consideration to applicants seeking flexible working patterns and those who have taken a career break. The University has a generous maternity policy, onsite childcare facilities, and offers a range of benefits to help ensure employees’ well-being and work-life balance. The University of Southampton is committed to sustainability and has been awarded the Platinum EcoAward.

Research areas