Improving Knowledge Graph Question Answering Accuracy

Summary & Methodology Analysis

The proposed method addresses the tendency of LLMs to generate ungrounded answers when querying knowledge graphs (KGs). Instead of relying solely on probabilistic generation, the system extracts lightweight constraints, such as type, relation, and exclusion, directly from the user prompt. It then processes these constraints using a three-valued logic system, which classifies candidate answers as satisfied, violated, or unknown. This logic is specifically designed to account for the reality of incomplete knowledge bases, allowing the system to handle missing data without discarding potentially correct answers prematurely. Candidates that result in a violated state are filtered out entirely, while remaining candidates are ranked using support scores based on the fraction of satisfied positive constraints. By applying exclusion constraints to specifically target and eliminate candidates that violate negated conditions, the approach realized a substantial increase in precision for Q2 queries, moving from 0.17 to 0.62. Across all test types, overall precision improved from 0.41 to 0.66, with recall remaining at 1.0. This demonstrates that the verification layer effectively cleans LLM output without inducing the false negatives typically associated with strict filtering. A primary limitation is that the evaluation was performed on controlled candidate sets rather than in an end-to-end KGQA pipeline. Furthermore, the system assumes the underlying knowledge graph is factually correct. If the graph contains incorrect triples or missing entities, the constraint evaluation process can inadvertently lead to false rejections or incorrect support for candidates. The paper does not specify the computational overhead of this verification step or the latency impact on query throughput.