MSc Research - Fault Tolerance Exploration in Large Language Models
Investigating the impact of bit-level parameter corruptions and resilience strategies in transformer-based models. The project is ongoing until August 2025.
Research Objectives:
Train and fine-tune a GraphCodeBERT-based common weakness enumeration classifier, using the buffer overflow code snippet dataset provided by the MSc supervisor, and evaluate its baseline inference ability in classifying CWEs.
Introduce bit flip attacks into various layers and parameters of the model architecture.
Conduct a comprehensive overview of what layers, weights and other parameters are most susceptible to bit flip attacks.
Explore various mechanisms of bit flip injections, including random bit flip attacks, progressive bit search and more.
Identify vulnerable parts of the transformer architecture and discuss how bit flip injection attacks can be neutralised and mitigated to maintain model inference.
Conduct analysis on various transformer models and various coding language datasets.
