Abstract
Automated Program Repair (APR) strives to automatically fix faulty software without human-intervention. Search-based APR iteratively generates possible patches for a buggy software, guided by the execution of the patched program on a test suite (i.e., a set of test cases). Search-based approaches have generally only used Boolean test case results (i.e., pass or fail), but recently more fined-grained fitness evaluations have been investigated with promising yet unsettled results. Using the most recent extension of the very popular Defects4J bug dataset, we conduct an empirical study using ARJA and ARJA-e, two state-of-the-art search-based APR systems using a Boolean and a non-Boolean fitness function, respectively. We aim to both extend previous results using new bugs from Defects4J v2.0 and to settle whether refining the fitness function helps fixing bugs present in large software.
In our experiments using 151 non-deprecated and not previously evaluated bugs from Defects4J v2.0, ARJA was able to find patches for 6.6% (\(\frac{10}{151}\)) of bugs, whereas ARJA-e found patches for 8% (\(\frac{12}{151}\)) of bugs. We thus observe only a small advantage in using the refined fitness function. This contrasts with the previous work using Defects4J v1.0.1 where ARJA was able to find adequate patches for 26.3% (\(\frac{59}{224}\)) of the bugs and ARJA-e for 47.3% (\(\frac{106}{224}\)). These results may indicate a potential overfitting of the tools towards the previous version of the Defects4J dataset.
Keywords
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bian, Z., Blot, A., Petke, J.: Refining fitness functions for search-based program repair. In: ICSE Workshops (2021)
Gazzola, L., Micucci, D., Mariani, L.: Automatic software repair: a survey. IEEE Trans. Softw. Eng. 45(1), 34–67 (2019)
Just, R., Jalali, D., Ernst, M.D.: Defects4J: a database of existing faults to enable controlled testing studies for Java programs. In: ISSTA, pp. 437–440 (2014)
Kechagia, M., Mechtaev, S., Sarro, F., Harman, M.: Evaluating automatic program repair capabilities to repair API misuses. IEEE Trans. Softw. Eng. (2021)
Le Goues, C., Nguyen, T., Forrest, S., Weimer, W.: GenProg: a generic method for automatic software repair. IEEE Trans. Softw. Eng. 38(1), 54–72 (2012)
Monperrus, M.: The living review on automated program repair. Technical report, hal-01956501, HAL (2018)
Motwani, M., Brun, Y.: Automatically repairing programs using both tests and bug reports. CoRR abs/2011.08340 (2020)
Smith, E.K., Barr, E.T., Goues, C.L., Brun, Y.: Is the cure worse than the disease? Overfitting in automated program repair. In: SIGSOFT FSE, pp. 532–543 (2015)
de Souza, E.F., Le Goues, C., Camilo-Junior, C.G.: A novel fitness function for automated program repair based on source code checkpoints. In: GECCO, pp. 1443–1450 (2018)
Trujillo, L., Villanueva, O.M., Hernandez, D.E.: A novel approach for search-based program repair. IEEE Softw. 38(4), 36–42 (2021)
Yuan, Y., Banzhaf, W.: ARJA: automated repair of java programs via multi-objective genetic programming. IEEE Trans. Softw. Eng. 46(10), 1040–1067 (2020)
Yuan, Y., Banzhaf, W.: Toward better evolutionary program repair: an integrated approach. ACM Trans. Softw. Eng. Methodol. 29(1), 5:1–5:53 (2020)
Zhu, Q., et al.: A syntax-guided edit decoder for neural program repair. In: ESEC/FSE (2021)
Acknowledgements
Funded by ERC 741278 and EPSRC EP/P023991/1.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Guizzo, G., Blot, A., Callan, J., Petke, J., Sarro, F. (2021). Refining Fitness Functions for Search-Based Automated Program Repair. In: O'Reilly, UM., Devroey, X. (eds) Search-Based Software Engineering. SSBSE 2021. Lecture Notes in Computer Science(), vol 12914. Springer, Cham. https://doi.org/10.1007/978-3-030-88106-1_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-88106-1_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88105-4
Online ISBN: 978-3-030-88106-1
eBook Packages: Computer ScienceComputer Science (R0)