Intelligent Manipulation of Deformed Objects via Visual–Tactile Fusion

About the project

About the Project Applications are invited for a self-funded, 3 year full-time or 6 year part-time PhD project. This project investigates intelligent robotic manipulation for deformable or soft objects by integrating visual and tactile sensing. Unlike rigid objects, deformable items (e.g., soft tissues, food products, or packaging materials) present significant challenges for robotic manipulation due to their unpredictable shapes and compliant properties. Relying on vision alone is often insufficient, as deformation and occlusion can distort perception. To address this, the project aims to develop a visual–tactile fusion framework that enables a robot to perceive, predict, and adapt its manipulating strategy in real time. The vision system provides global context—object geometry, pose, and deformation state—while tactile sensors supply local contact feedback such as pressure distribution, slip, and texture. By fusing these complementary modalities through deep learning or probabilistic models, the robot can achieve robust and adaptive control even under uncertain or variable conditions. The outcomes will advance the field of soft object manipulation and contribute to applications in healthcare, agriculture, and manufacturing. The PhD will be based in the School of Electrical and Mechanical Engineering and will be supervised by Dr Xin Zhang . The work on this project will: Perception of Deformable Objects. Combines RGB/depth vision with high-resolution tactile sensing to estimate deformation and contact states dynamically during manipulation. Multimodal Sensor Fusion. Develops a versatile fusion model using advanced machine learning algorithms (such as diffusion model, reinforcement learning) to integrate visual and tactile data, enhancing object recognition, contact estimation, and grasp stability prediction. Simulation training and experiment verification. The simulation will be used to train the algorithm. A large amount of deformed objects will be used to test the developed smart manipulation system Project description Manipulating deformable objects remains one of the most challenging problems in modern robotics. Unlike rigid items, deformable objects such as human tissues, food products, or flexible components change shape during manipulation, making perception and control highly uncertain. Students will gain hands-on experience in robot programming, computer vision, and deep learning, using collaborative robotic platforms and tactile sensor technologies available at the University of Portsmouth. Some related achievements can be found as follows: Efficient template-based robotic sorting with one-shot multi-object pose estimation algorithm[C]//2024 30th International Conference on Mechatronics and Machine Vision in Practice (M2VIP). IEEE, 2024: 1-6. Proximal policy optimization with model-based methods[J]. Journal of Intelligent & Fuzzy Systems, 2022, 42(6): 5399-5410. General admissions criteria You'll need a good first degree from an internationally recognised university (minimum upper second class or equivalent, depending on your chosen course) or a Master’s degree in Robotics, Electrical Engineering, Mechatronics, Mechanical Engineering, Computer Science, or a related area. In exceptional cases, we may consider equivalent professional experience and/or Qualifications. English language proficiency at a minimum of IELTS band 6.5 with no component score below 6.0. Specific candidate requirements The candidate can demonstrate their ability and aptitude for researching work, with a degree or transcripts of courses relevant to Mechanical Engineering and/or Computer Science. Programming and analytical skills (e.g., Python, ROS, control systems, or machine learning frameworks) Any background and/or hands-on experience in robots and machine vision is an advantage. How to Apply We’d encourage you to contact Dr Xin Zhang ( xin.zhang@port.ac.uk ) to discuss your interest before you apply, quoting the project code. When you are ready to apply, please follow the ' Apply now ' link on the Electronic Engineering PhD subject area page and select the link for the relevant intake. Make sure you submit a personal statement, proof of your degrees and grades, details of two referees, proof of your English language proficiency and an up-to-date CV. Our ‘ How to Apply ’ page offers further guidance on the PhD application process. When applying please quote project code: SEM10370526

Research areas