Source Learning LLM Agents Persistent Sources

From Knowledge Access to Source Learning: Developing Source-Specific Competence

Lucheng Fu1, Kejing Xia1, Yiyang Wang1, Yiqiao Jin1, Jinjin He1, Xiyuan Yang2, Haoxin Liu1, Ye Yu2, Haibo Jin2, Yijia Xiao3, Wenke Lee1†, B. Aditya Prakash1†, Haohan Wang2†
1 Georgia Institute of Technology    2 University of Illinois at Urbana-Champaign    3 University of California, Los Angeles
† Corresponding authors
13 / 15
settings where SourceLearn is best
Up to +22.6
points over Hybrid RAG
5
benchmarks: documents, code, APIs, apps
3
LLM backbones
Abstract

Large language model (LLM) agents increasingly rely on persistent external sources to solve sequences of knowledge-intensive tasks. Existing methods improve how source content is accessed and organized, while agent-memory systems preserve reusable knowledge from prior interactions, but repeated use of the same source is still largely treated as repeated access rather than an opportunity to progressively improve understanding of that source. We study source learning: developing reusable source-specific competence over a persistent authoritative source. We represent this competence with a persistent source model that captures reusable understanding of the source, including how its knowledge is structured, interpreted, and applied. To construct and progressively refine such models, we propose SourceLearn, which combines two complementary learning mechanisms. Self-Directed Source Learning identifies what remains incompletely understood and adaptively revisits the source, while Task-Guided Source Learning uses downstream experience to reveal local representational gaps and recurring needs in how source knowledge should be organized. In both cases, learning signals determine what should be reconsidered, while persistent updates are reconstructed from the authoritative source. Across five benchmarks and three LLM backends, SourceLearn achieves the best performance in 13 of 15 settings, with gains of up to 22.6 points over Hybrid RAG and substantial overall improvements over static source representations and experience-based memory baselines.

Problem

Repeated access is not learning.

Retrieval and agent memory make a source easier to access and past experience easier to reuse. When many tasks depend on the same source, the agent can do more: build source-specific competence, a reusable understanding of how the source's knowledge is structured, interpreted and applied, while the source itself stays authoritative.

Knowledge access versus source learning

Knowledge access retrieves evidence per task and re-derives how to use it every time (left). Source learning accumulates reusable understanding across tasks while keeping direct access to the source (right).

Challenge I

What should be learned from a source before future tasks are known?

Challenge II

How can task feedback improve source competence beyond the task that produced it, without memorizing answers?

The source model

A persistent, revisable state M of what has been learned about the source, used next to retrieval: ŷ = F(q, R(D,q), M).

Framework

A source model that learns through study and use

Three steps, one rule: learning signals decide what to reconsider; the source writes the update. Every persistent change is reconstructed from source evidence and must preserve previously supported meaning.

Overview of SourceLearn

Overview. Construction builds M0; Self-Directed Source Learning deepens and connects it into MS; Task-Guided Source Learning refines it into MT; at task time the activated model is combined with retrieved evidence.

1

Source Model Construction

Entity-centered reading of the source yields a provisional model M0: entities, mechanisms, conditions, procedures, exceptions and relations. Look-up details stay in the source.

2

Self-Directed Source Learning

Inspect → Study → Consolidate. Reread the source against the model, plan Deepen / Connect study actions where it falls short, then consolidate into MS.

3

Task-Guided Source Learning

Guidance tasks expose gaps. Local refinement rereads the region a failure points to; cross-task lessons become a representation policy Π1 that recalibrates the whole model into MT. Answers are never stored.

Learning
D  →  M0  →  MS  →  MT
Grounded reconstruction
Inspect → Reconstruct → GroundApply
Task time
ŷ = F(q, R(D, q), AB(M, q))
Results

Consistent gains across source types and LLM backbones

Five benchmarks (document QA, code QA, tool use, interactive environments), three backbones, disjoint guidance and test tasks. SourceLearn receives the same retrieved evidence as Hybrid RAG plus its activated source model.

GPT-5.6-Luna · 5-benchmark mean
+14.3 vs Hybrid RAG
gpt-oss-120b · 5-benchmark mean
+4.9 vs Hybrid RAG
DeepSeek-V4.1-Flash · 5-benchmark mean
+13.4 vs Hybrid RAG
GPT-5.6-Luna · APIBench
70.4 +22.6 vs Hybrid RAG
Backbone Method Document QA Code QA Tool Use Interactive Env.
MultiDoc2DialNarrativeQA SWE-QA APIBench AppWorld
GPT-5.6-LunaHybrid RAG 64.966.056.947.872.0
RAPTOR 71.2+6.368.1+2.150.8−6.161.9+14.175.6+3.6
HippoRAG 2 66.3+1.466.6+0.662.7+5.855.2+7.477.4+5.4
AWM 65.1+0.262.1−3.956.7−0.262.8+15.066.7−5.3
SourceLearn 77.6+12.780.2+14.269.5+12.670.4+22.681.5+9.5
gpt-oss-120bHybrid RAG 64.545.652.560.330.4
RAPTOR 61.5−3.040.4−5.232.8−19.754.7−5.626.8−3.6
HippoRAG 2 66.7+2.245.7+0.156.7+4.256.0−4.333.9+3.5
AWM 64.1−0.443.2−2.451.5−1.063.2+2.928.6−1.8
SourceLearn 68.7+4.256.7+11.153.4+0.971.8+11.527.4−3.0
DeepSeek-V4.1-FlashHybrid RAG 63.364.245.967.183.9
RAPTOR 73.2+9.960.4−3.836.8−9.169.3+2.288.7+4.8
HippoRAG 2 67.3+4.062.0−2.255.7+9.869.6+2.586.9+3.0
AWM 63.7+0.461.4−2.847.4+1.570.8+3.783.9+0.0
SourceLearn 81.3+18.076.2+12.061.6+15.782.2+15.189.9+6.0

Task performance (%); subscripts are changes relative to Hybrid RAG. Bold: best. Underlined: second-best.

Analysis

Both mechanisms matter, and the model is more than compression.

What each mechanism contributes, what the source model comes to represent, and how it holds up when retrieval fails.

Ablation of SourceLearn

Ablation (GPT-5.6-Luna, two sources per QA benchmark): learning stages (left), Task-Guided pathways (right).

Complementary mechanisms

MT is best on every benchmark; dropping either Self-Directed or Task-Guided learning lowers accuracy, and M0 is lowest.

Two pathways, different signals

Failures repair local gaps; recurring lessons reshape the whole model. Removing either hurts, removing both hurts more.

Beyond compression

Learning shifts the model from look-up content to rules and procedures: conditioned units rise from ~30% to 65–82%, coverage of the claims test questions need rises from 23.2% to 40.0%, and accuracy goes from 67.5% with none of them covered to 86.6% with all of them covered.

Source learning beyond compression

From M0 to MS to MT: (a) what is represented, (b) explicit conditions, (c) coverage of required claims, (d) accuracy vs. coverage.

Robustness to missing retrieved evidence

Robustness (MultiDoc2Dial, GPT-5.6-Luna): (a) fewer retrieved elements, (b) gold document removed at k = 8.

Robust to missing evidence

With no retrieval, the source model alone matches Hybrid RAG with 8 elements. Removing the gold document costs Hybrid RAG about 25 points, SourceLearn about 8.

Transfers to unseen regions

+14.9 points over Hybrid RAG on regions touched by guidance tasks, +12.1 on regions no guidance task touched.

Grounded

Contradictions with cited evidence stay ≤1%; 93–100% of claims dropped during rewriting remain represented.

BibTeX

If you find this work useful, please cite

@article{fu2026knowledge,
  title={From Knowledge Access to Source Learning: Developing Source-Specific Competence},
  author={Fu, Lucheng and Xia, Kejing and Wang, Yiyang and Jin, Yiqiao and He, Jinjin and Yang, Xiyuan and Liu, Haoxin and Yu, Ye and Jin, Haibo and Xiao, Yijia and others},
  journal={arXiv preprint arXiv:2610.02150},
  year={2026}
}