- author = "Karine Even-Mendoza and Alexander Brownlee and Alina Geiger and Carol Hanna and Justyna Petke and Federica Sarro and Dominik Sobania",
- title = "{LLM}-Guided Genetic Improvement: Envisioning Semantic Aware Automated Software Evolution",
- howpublished = "arXiv preprint arXiv:2508.18089",
- year = "2025",
- month = "25 " # aug,
- keywords = "genetic algorithms, genetic programming, genetic improvement, Gin, Large language models, ANN, AI, Machine Learning (ML), PatchCat, software errors, optimisation, patchDiff, briefSum",
- eprint = "2508.18089",
- archiveprefix = "arXiv",
- primaryclass = "cs.SE",
-
URL = "
https://arxiv.org/abs/2508.18089",
-
code_url = "https://doi.org/10.5281/zenodo.15834984",
- size = "5 pages",
-
abstract = "Genetic Improvement (GI) of software automatically
creates alternative software versions that are improved
according to certain properties of interests (e.g.,
running-time). Search-based GI excels at navigating
large program spaces, but operates primarily at the
syntactic level. In contrast, Large Language Models
(LLMs) offer semantic-aware edits, yet lack
goal-directed feedback and control (which is instead a
strength of GI).
As such, we propose the investigation of a new research line on AI-powered GI aimed at incorporating semantic aware search. We take a first step at it by augmenting GI with the use of automated clustering of LLM edits.
We provide initial empirical evidence that our proposal, dubbed PatchCat, allows us to automatically and effectively categorize LLM-suggested patches. PatchCat identified 18 different types of software patches and categorized newly suggested patches with high accuracy. It also enabled detecting NoOp edits in advance and, prospectively, to skip test suite execution to save resources in many cases. These results, coupled with the fact that PatchCat works with small, local LLMs, are a promising step toward interpretable, efficient, and green GI.
We outline a rich agenda of future work and call for the community to join our vision of building a principled understanding of LLM-driven mutations, guiding the GI search process with semantic signals.",
-
notes = "Also known
\cite{evenmendoza2025llmguidedgeneticimprovementenvisioning}
'Mistral generated patches, and JUnit4 and JCodec projects'
'Sentence-BERT (MiniLM) ... Normalised Mutual Information (NMI)s
llama3
TABLE I: Category (#) & Description.
0 Added (some arbitrary) code from GitHub
1 No change
2 Modified a comment (add/remove/edit)
3 Deleted blocks in a method (all/most/some)
4 Duplicate code
5 Modifications to return statements (add/remove/edit)
6 Changes to method names
7 Changed data types or type usage and generics
8 Includes inlining of implementations (sort, methods...)
9 Added exception-handling constructs (unreachable or reachable)
10 Added extra brackets
11 Added synchronization logic
12 Modified Variable/Class/Object Name
13 Modified Control Flow Structure
14 Modified Object/Primitive Creation or Initialization
15 Split a statement into multiple lines
16 Arithmetic manipulation (Boolean variable or expression manipulations)
17 Added dead code
King's College London, UK",
