Knowledge distillation in the QAOA setting for advantage on quantum hardware

In this work, we investigate the application of Knowledge Distillation (KD) to the Quantum Approximate Optimization Algorithm (QAOA) setting, introducing a framework to distill knowledge from a Multi-angle QAOA teacher to a standard, single-layer QAOA student. We propose three distillation schemes: count-based KD, distillation using MSE, and distillation with Huber loss for enhanced robustness. Numerical experiments on diverse graph families, including complete graphs, Erdős–Rényi graphs, complete binary trees, and random 3-regular graphs, suggest that MSE and Huber-based distillation can improve the student’s performance, while count-based KD exhibits higher variance and limited generalization. Our results indicate that KD can enable a compression of the QAOA parameter space while often maintaining competitive solution quality, providing a practical avenue for efficient deployment on Noisy Intermediate-Scale Quantum hardware with improved training stability.