Welcome!
To use the personalized features of this site, please log in or register.
If you have forgotten your username or password, we can help.
My Menu
Saved Items

TACAS 06

An approach for experimentally evaluating effectiveness and efficiency of coverage criteria for software testing

Atul GuptaContact Information and Pankaj Jalote1, 2 Contact Information

(1)  Indian Institute of Technology, Kanpur, 208016, India
(2)  Present address: Indian Institute of Technology Delhi, Delhi, 110016, India

Published online: 8 January 2008

Abstract  Experimental work in software testing has generally focused on evaluating the effectiveness and effort requirements of various coverage criteria. The important issue of testing efficiency has not been sufficiently addressed. In this paper, we describe an approach for comparing the effectiveness and efficiency of test coverage criteria using mutation analysis. For each coverage criterion under study, we generate multiple coverage-adequate minimal test suites for a test-program from a test-pool, which are then executed on a set of systematically generated program mutants to obtain the fault data. We demonstrate the applicability of the proposed approach by describing the results of an experiment comparing the three code-based testing criteria, namely, block coverage, branch coverage, and predicate coverage. Our results suggest that there is a trade-off between effectiveness and efficiency of a coverage criterion. Specifically, the predicate coverage criterion was found to be most effective but least efficient whereas the block coverage criterion was most efficient but least effective. We observed high variability in the performance of block test suites whereas branch and predicate test suites were relatively consistent. Overall results suggest that the branch coverage criterion performs consistently with good efficiency and effectiveness, and it appears to be the most viable option for code-based control flow testing.

Keywords  Coverage based testing - Coverage criteria - Test case generation - Experimental evaluation - Mutation operators - Statistical analysis


Contact Information Atul Gupta (Corresponding author)
Email: atulkg@gmail.com

Contact Information Pankaj Jalote
Email: jalote@cse.iitd.ac.in
Fulltext Preview (Small, Large)
Image of the first page of the fulltext

References secured to subscribers.



Export this article
Export this article as RIS | Text
 
Remote Address: 38.107.191.112 • Server: mpweb02
HTTP User Agent: CCBot/1.0 (+http://www.commoncrawl.org/bot.html)