# Experimental results An introduction to GABench and its test questions, followed by graph data agent success rates across four task families and construction query cost reductions relative to PK/FK rules. ## Paper and question source: GABench The four graph-analysis experiments use questions from our work, GABench. Its 10,400 tasks assess agents’ ability to plan, use graph tools, and answer from their results. - [GABench: A Comprehensive Benchmark for Evaluating LLM Agents on Graph Analysis Tasks · arXiv:2608.01684](https://arxiv.org/abs/2608.01684) ## Overview of test questions | Task family | What the questions ask | | --- | --- | | Graph retrieval | Retrieve node attributes, edges, and graph statistics. | | Graph theory | Compute centrality, shortest paths, and global structural properties. | | Graph machine learning | Perform node classification, link prediction, and graph classification. | | Open-ended graph question answering | Use graph evidence to develop insights, predictions, and explanations. | - NAG: Numerical-Attribute Graph, with numerical node attributes. - TAG: Text-Attributed Graph, with textual node attributes. - TPG: Text-Paired Graph, paired with a graph-level text description. - [Task definitions: Section 3.3; graph types: Section 2.2](https://arxiv.org/html/2608.01684v1#S3.SS3) ## Experimental setup and reading the results The four graph-analysis experiments use GLM-5-turbo as the base model; rows identify the agent harness. A model-only row in the source results denotes OpenClaw. These four tables report only SR (task success rate), as a percentage with two decimal places; higher is better. Each SR column is ranked independently: the highest score is bold, the second-highest score is underlined, and ties share the same marking. Retrieval and theory compare OpenClaw and iGraph; machine learning and open-ended question answering compare Hermes, Claude Code, OpenClaw, and iGraph. - [Construction experiments: comparison with PK/FK rule-based construction](/en/experiments/#graph-construction) ## Graph retrieval Retrieval SR is reported at the node, edge, and graph levels. **Graph retrieval · GLM-5-turbo · SR (%) ↑** | Agent harness | NAG · Node | NAG · Edge | NAG · Graph | TAG · Node | TAG · Edge | TAG · Graph | TPG · Node | TPG · Edge | TPG · Graph | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | OpenClaw | 8.33 | 6.25 | 6.67 | 72.91 | 18.18 | 27.27 | 60.00 | 11.11 | 42.86 | | iGraph | 31.00 | 94.74 | 18.67 | 100.00 | 100.00 | 82.50 | 100.00 | 100.00 | 100.00 | ## Graph theory Graph-theory SR is reported at the node, edge, and graph levels. **Graph theory · GLM-5-turbo · SR (%) ↑** | Agent harness | NAG · Node | NAG · Edge | NAG · Graph | TAG · Node | TAG · Edge | TAG · Graph | TPG · Node | TPG · Edge | TPG · Graph | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | OpenClaw | 24.00 | 26.81 | 7.40 | 61.02 | 9.52 | 21.66 | 28.00 | 13.33 | 10.49 | | iGraph | 37.53 | 38.14 | 45.00 | 94.44 | 88.24 | 88.89 | 100.00 | 100.00 | 100.00 | ## Graph machine learning NC denotes node classification, LP link prediction, and GC graph classification. Scores measure the agent’s success rate (SR) on the corresponding tasks. **Graph machine learning · GLM-5-turbo · SR (%) ↑** | Agent harness | NAG · NC | NAG · LP | TAG · NC | TAG · LP | TPG · GC | | --- | --- | --- | --- | --- | --- | | Hermes | 61.54 | 90.90 | 54.35 | 87.18 | 7.14 | | Claude Code | 95.23 | 78.57 | 86.79 | 89.53 | 88.89 | | OpenClaw | 26.66 | 65.18 | 19.36 | 36.77 | 26.74 | | iGraph | 90.00 | 100.00 | 95.00 | 100.00 | 85.00 | ## Open-ended graph question answering Insight and Prediction identify the two open-ended task categories. Both are compared using SR. **Open-ended graph question answering · GLM-5-turbo · SR (%) ↑** | Agent harness | NAG · Insight | NAG · Prediction | TAG · Insight | TAG · Prediction | TPG · Insight | TPG · Prediction | | --- | --- | --- | --- | --- | --- | --- | | Hermes | 0.00 | 8.22 | 1.28 | 15.12 | 16.28 | 8.11 | | Claude Code | 7.63 | 3.70 | 18.59 | 20.00 | 29.63 | 20.69 | | OpenClaw | 7.65 | 3.22 | 13.56 | 10.47 | 14.29 | 12.50 | | iGraph | 25.00 | 8.60 | 55.00 | 42.50 | 35.00 | 21.00 | ## Graph construction agent The baseline graph is built using primary-key and foreign-key (PK/FK) rules. Results compare query cost after the construction agent optimizes the graph schema. Optimization queries guide the search and evaluation of candidate graph schemas; held-out queries assess improvements on a separate query set. **Construction agent versus PK/FK rule-based construction · Baseline: PK/FK** | Dataset | Optimization query cost reduction | Held-out query cost reduction | | --- | --- | --- | | Amazon | 27.33% | 53.20% | | AVS | 14.80% | 20.71% | | MAG | 11.79% | 19.91% | | StackExchange | 8.40% | 6.74% | For each dataset, select the supplied result with lower costs on both optimization and held-out queries: the first result set for Amazon and AVS, and the second for MAG and StackExchange. Both reductions in a row come from the same result set and retain the reported values. This is a per-dataset best-result summary. Percentages indicate cost reduction relative to PK/FK rule-based construction; larger reductions are better. Rows represent different datasets and are not ranked against one another. - [How the construction agent optimizes and validates graph schemas](/en/construction/)