An analysis of the relationship between conditional entropy and failed error propagation in software testing

Conference paper


Androutsopoulos, K., Clark, D., Dan, H., Hierons, R. and Harman, M. 2014. An analysis of the relationship between conditional entropy and failed error propagation in software testing. 36th International Conference on Software Engineering, ICSE '14. Hyderabad, India 31 May - 07 Jun 2014 Association for Computing Machinery (ACM). pp. 573-583 https://doi.org/10.1145/2568225.2568314
TypeConference paper
TitleAn analysis of the relationship between conditional entropy and failed error propagation in software testing
AuthorsAndroutsopoulos, K., Clark, D., Dan, H., Hierons, R. and Harman, M.
Abstract

Failed error propagation (FEP) is known to hamper software testing, yet it remains poorly understood. We introduce an information theoretic formulation of FEP that is based on measures of conditional entropy. This formulation considers the situation in which we are interested in the potential for an incorrect program state at statement s to fail to propagate to incorrect output. We define five metrics that differ in two ways: whether we only consider parts of the program that can be reached after executing s and whether we restrict attention to a single program path of interest .We give the results of experiments in which it was found that on average one in 10 tests suffered from FEP, earlier studies having shown that this figure can vary significantly between programs. The experiments also showed that our metrics are well-correlated with FEP. Our empirical study involved 30 programs, for which we executed a total of 7,140,000 test cases. The results reveal that the metrics differ in their performance but the Spearman rank correlation with failed error propagation is close to 0.95 for two of the metrics. These strong correlations in an experimental setting, in which all information about both FEP and conditional entropy is known, open up the possibility in the longer term of devising inexpensive information theory based metrics that allow us to minimise the effect of FEP.

Conference36th International Conference on Software Engineering, ICSE '14
Page range573-583
Proceedings TitleICSE 2014: Proceedings of the 36th International Conference on Software Engineering
ISBN
Hardcover9781450327565
PublisherAssociation for Computing Machinery (ACM)
Publication dates
Print31 May 2014
Publication process dates
Deposited28 Nov 2014
Output statusPublished
Web address (URL)http://doi.acm.org/10.1145/2568225.2568314
Digital Object Identifier (DOI)https://doi.org/10.1145/2568225.2568314
LanguageEnglish
Permalink -

https://repository.mdx.ac.uk/item/84y29

  • 57
    total views
  • 0
    total downloads
  • 0
    views this month
  • 0
    downloads this month

Export as

Related outputs

Correctly slicing extended finite state machines
Amtoft, T., Androutsopoulos, K. and Clark, D. 2020. Correctly slicing extended finite state machines. in: Di Pierro, A., Malacaria, P. and Nagarajan, R. (ed.) From lambda calculus to cybersecurity through program analysis - essays dedicated to Chris Hankin on the occasion of his retirement Switzerland Springer International Publishing.
Evelyne Lutton, Nathalie Perrot, Alberto Tonda: Evolutionary algorithms for food science and technology, Wiley, 2016, 182 pp, ISBN: 978-1-119-13683-5 [Book review]
Androutsopoulos, K. 2018. Evelyne Lutton, Nathalie Perrot, Alberto Tonda: Evolutionary algorithms for food science and technology, Wiley, 2016, 182 pp, ISBN: 978-1-119-13683-5 [Book review]. Genetic programming and Evolvable Machines. https://doi.org/10.1007/s10710-018-9335-2
MIRTO: an open-source robotic platform for education
Androutsopoulos, K., Aristodemou, L., Boender, J., Bottone, M., Currie, E., El-Aroussi, I., Fields, B., Gheri, L., Gorogiannis, N., Heeney, M., Micheletti, M., Loomes, M., Margolis, M., Petridis, M., Piermarteri, A., Primiero, G., Raimondi, F. and Weldin, N. 2018. MIRTO: an open-source robotic platform for education. 3rd European Conference on Software Engineering Education. Seeon, Germany 14 - 15 Jun 2018 Association for Computing Machinery (ACM). pp. 55-62 https://doi.org/10.1145/3209087.3209106
A racket-based robot to teach first-year computer science
Androutsopoulos, K., Gorogiannis, N., Loomes, M., Margolis, M., Primiero, G., Raimondi, F., Varsani, P., Weldin, N. and Zivanovic, A. 2014. A racket-based robot to teach first-year computer science. 7 th European Lisp Symposium. IRCAM, Paris, France 05 - 06 May 2014 pp. 54-61
An alternative characterization of weak order dependence
Amtoft, T., Androutsopoulos, K., Clark, D., Harman, M. and Li, Z. 2010. An alternative characterization of weak order dependence. Information Processing Letters. 110 (21), pp. 939-943. https://doi.org/10.1016/j.ipl.2010.07.023
Model projection: simplifying models in response to restricting the environment
Androutsopoulos, K., Binkley, D., Clark, D., Gold, N., Harman, M., Lano, K. and Li, Z. 2011. Model projection: simplifying models in response to restricting the environment. Proceedings of the 33rd International Conference on Software Engineering, ICSE 2011. Waikiki, Honolulu, HI, USA 21 - 28 May 2011 pp. 291-300
State-based model slicing: a survey
Androutsopoulos, K., Clark, D., Harman, M., Krinke, J. and Tratt, L. 2013. State-based model slicing: a survey. ACM Computing Surveys. 45 (4), p. 53. https://doi.org/10.1145/2501654.2501667
Model projection: simplifying models in response to restricting the environment
Androutsopoulos, K., Binkley, D., Clark, D., Gold, N., Harman, M., Lano, K. and Li, Z. 2011. Model projection: simplifying models in response to restricting the environment. in: Proceedings of the 33rd International Conference on Software Engineering - ICSE '11 ACM. pp. 291-300
Amorphous slicing of extended finite state machines
Androutsopoulos, K., Clark, D., Harman, M., Hierons, R., Li, Z. and Tratt, L. 2013. Amorphous slicing of extended finite state machines. IEEE Transactions on Software Engineering. 39 (7), pp. 892-909. https://doi.org/10.1109/TSE.2012.72
A theoretical and empirical study of EFSM dependence.
Androutsopoulos, K., Gold, N., Harman, M., Li, Z. and Tratt, L. 2009. A theoretical and empirical study of EFSM dependence. IEEE Computer Society. pp. 287-296 https://doi.org/10.1109/ICSM.2009.5306309
Control dependence for extended finite state machines.
Androutsopoulos, K., Clark, D., Harman, M., Li, Z. and Tratt, L. 2009. Control dependence for extended finite state machines. Fundamental Approaches to Software Engineering (FASE '09). Springer. https://doi.org/10.1007/978-3-642-00593-0_15