Using LLMs to Improve Compiler Optimization

Summary & Methodology Analysis

The research investigates whether LLMs can augment traditional compiler optimization pipelines (specifically -O3, LTO, and PGO). The core methodology involves identifying semantic facts and program equivalences, such as data-structure invariants or low-level assumptions, from heterogeneous code contexts. Once identified, the models generate source-level artifacts like qualifiers, guards, or code rewrites that preserve the program contract while providing opportunities for performance gains that conventional compilers overlook during standard optimization passes. The paper introduces SeGaBench, an executable benchmark suite designed to evaluate these capabilities through a protocol of blind inference followed by independent compilation and profiling. The experimental setup utilizes a variety of models including GPT-5.4 mini, GPT-5.6 Sol, DeepSeek-V4 Pro, Llama 3.3 70B Instruct Turbo, and Ternary Bonsai 27B across several workloads like HPCG, LAMMPS, LULESH, miniFE, RAJAPerf, and XSBench. The evaluation shows a performance discrepancy between synthetic and real-world cases, with success rates dropping from 86.6% to 67.0% respectively. Limitations are notable, as the study is constrained to a single hardware and compiler environment. Furthermore, the selection of real-world cases focused on reproducibility and oracle speedup, which may not capture the full spectrum of optimization patterns present in production environments. Correctness is also strictly tied to the predefined workload contracts for each benchmark entry, meaning these results cannot necessarily be generalized across all software deployment scenarios.