LLM-Driven Root Cause Analysis for Distributed System Incidents: A Reproducible Framework

Abstract

In Distributed Systems, Root Cause Analysis (RCA) is a complex and time-sensitive task that demands correlating a variety of telemetry data such as logs, metrics, and distributed traces. In traditional contexts these activities are fundamentally dependent on the skills of Site Reliability Engineers (SREs) and incident diagnosis can be time-consuming and challenging, and becomes increasingly difficult as applications move to increasingly complex cloud-native contexts. The opportunities for automated reasoning by recent advances in Large Language Models (LLMs) are also accompanied by methods for automated RCA based on LLMs, which are often limited to unstructured summaries and have the potential for hallucination and unreliable conclusions. VerifiedRCA proposes an extensible and reproducible approach combining LLM reasoning, structured telemetry retrieval and tool-call verification to ensure trustworthy automated incident diagnosis. The proposed architecture consists of four layers of a pipeline: telemetry ingestion, retrieval-augmented context construction, hypothesis generation and evidence-based verification. Synthetic scenarios were injected (120 in total) into a realistic micro services environment built using Google Hipster Shop, Istio service mesh and Chaos Mesh fault injection for evaluation. Using the accuracy, time-to-diagnosis and explanation quality metrics, and performance was compared to rule-based systems, classical machine learning approaches, and ungrounded LLM baselines. The results show that grounding the LLM's reasoning with telemetry verification greatly enhances the diagnostic accuracy, with higher precision, more rapid convergence, and a decrease in unsupported reasoning paths. This study provides a clear RCA framework and an open benchmark that can facilitate future research and development in operational intelligence with the help of LLMs and support reproducible research.