Assessing the quality of behavior-driven development scenarios using BERT

This research investigates the application of BERT (Bidirectional Encoder Representations from Transformers), a state-of-the-art Machine Learning (ML) model for Natural Language Processing (NLP), to improve the quality assessment of Behavior-Driven Development (BDD) scenarios. BDD is a widely used technique in Agile software development, which clarifies feature behavior by defining test scenarios for user stories. While user stories are effective for outlining software requirements, they often lack detailed validation criteria, and the manual evaluation of BDD scenarios can be time-consuming and subjective, leading to inconsistencies, rework, and project delays. To address these challenges, this study explores how BERT can enhance scenario evaluation by leveraging its advanced language understanding capabilities to detect ambiguities and inconsistencies more accurately than traditional methods. Precision, which measures the accuracy of the model's correct predictions, was 70.1 %, indicating how often the model's identified defects were truly defects. Recall, the measure of how many relevant defects the model successfully identified, reached 80.5 %. The F1 score, a balance between precision and recall, was 75.3 %, demonstrating BERT's effectiveness in handling imbalanced data. These results suggest that BERT significantly improves both the objectivity and efficiency of BDD scenario evaluation, offering a valuable tool for enhancing software development processes. This research contributes to the integration of NLP in Agile software development, providing a foundation for future exploration of AI-driven solutions for improving software quality and requirement documentation.