Artificial intelligence model security

Project summary

Artificial intelligence (AI)-based cyber security has emerged as a promising solution for cyber defence in the past decade because of the proliferation of deep-learning-based deep neural network models. However, the adversaries and the hackers have also discovered some vulnerabilities in AI-based systems. Then AI-model security has arisen as a new problem. Not only attackers but also researchers have developed different attacks against AI models. In the beginning, they use vulnerabilities and issues such as poor access control to servers hosting data, exploitable bugs and logic flaws in software applying a particular AI model, and lack of sufficient logging and monitoring of AI-model activity to carry out the attacks.

However, new and contemporary security attacks against AI models have been derived from the nature of the mathematics of AI itself. The adversary may fool the model, skew the model by carefully poisoning the input data or use carefully crafted queries to steal sensitive personal data used to train the model and sometimes even the model parameters.

In this project, the AI security problem will be addressed in the context of cyber security. The main goal is to provide solutions to protect AI models from cyber attacks.

Research objectives

This project is supported by endpoint protection specialist CrowdStrike . The PhD candidate will work closely with the research team in CrowdStrike.

• To investigate good security practices in securing the full range of infrastructure that AI development requires and develop guidelines for secure AI modelling
• To investigate AI models vulnerabilities and to develop novel monitoring techniques for real time input data to AI models to mitigate poisoning attacks
• To investigate how explainable AI can be used in protecting AI models
• To design secure AI models against current and zero-day attacks

Research centre

Data Science Research Centre

Entry requirements

For our PhD programmes, we normally expect you to have a first-class or upper-second (2:1) honours degree and preferably a masters degree from a UK university university or qualifications that we consider to be equivalent.

International students may also need to meet our English language requirements. Find out more about our entry requirements for international students.

Project-specific requirements must align with the University’s standard requirements.

How to apply

Please contact Dr Abid Khan in the first instance for more information on how to apply.

The University has four starting points each year for MPhil/PhD programmes (September, January, March and June). Applications should be made at least three months before you would want to start your programme. Please note that, if you require a visa, additional time will be required.

Funding

Self-funded by student. There is a range of options that may be available to you to help you fund your PhD.

Supervisors

Dr Ovidiu Bagdasar is the Erasmus Coordinator for Mathematics and Computing. His research in Discrete Mathematics, Optimisation, and Maths Anxiety has been disseminated in numerous international journals and conferences. Ovidiu also works with colleagues and technology companies to improve standards in mathematics education within the University, and beyond.

We are looking for ambitious and talented students to join our dedicated research community where we will support you on your own research journey through to graduation. These pages give you all the i...

If you decide to design your own PhD project, you will need to write a research proposal that will form part of your application. Before applying, you should review the critical literature in your sub...

Find out what the tuition fees are for PhDs and MPhils at Derby, as well as what sources of funding there are to help pay for them.

Research areas