Lab
Finding a decision inside a long document
Short answer
A search can retrieve the right meeting record without exposing the decision that answers the question. The document appears among the results, while the useful paragraph remains buried among unrelated topics. That gap led us to investigate retrieval at a finer scale: the passage.

The right document is only part of the answer
Consider a meeting record covering suppliers, working hours and an exception to a maintenance rule. The question concerns only the exception. The whole document may have a weak match with that question, even though a few sentences provide a precise answer.
Finding the document does not guarantee that those sentences reach the assistant. Context has a limited budget. Several complete memories can consume that budget before the decisive paragraph appears. Returning only the beginning of each memory leaves the same problem unresolved.
Passage retrieval tries to match the unit of retrieval to the unit of the question. It must also preserve the information that makes an excerpt meaningful: its source, version, date and qualifications.
Testing the idea on more demanding material
An initial Lab campaign examined decisions buried in synthetic meeting records. Passage selection helped substantially when a decision used different wording from the question. Where the wording matched, a simpler lexical excerpt could already work well.
A second campaign used an unseen corpus with continuous prose, tables, decisions spread across paragraphs and uncertainty markers. It compared complete memories limited by the context budget, search excerpts and selected passages.
For paraphrased prose questions in that confirmation set, the main model’s correct-answer rate reached 0.63 with selected passages, compared with 0.06 for complete memories and 0.00 for excerpts alone. These are results for defined synthetic cases, rather than a general product success rate.
That distinction matters. A controlled corpus makes mechanisms visible, but it does not establish how often each mechanism will occur in ordinary organisational documents.
A precise excerpt can still mislead: scope and limits
Greater focus introduces a risk. A number can appear in one paragraph while “nothing has been approved” appears much farther away. Returning the number without its qualification creates incomplete context, even when the quotation itself is accurate.
In the confirmation set, distant status markers explained the observed losses of context. Keeping an adjacent paragraph was not always enough. A decision distributed across several passages creates a similar problem: retrieving one part does not establish completeness.
Hidden malicious instructions raised a further issue. A selected passage can expose a sentence that the previous presentation never reached. One of the smaller models followed the embedded instruction in two of the four relevant cases. Better retrieval, by itself, is therefore insufficient evidence of improved safety.
This is a useful distinction between retrieval quality and the authority given to retrieved material. Making text easier to find does not make it trustworthy or entitled to instruct the assistant.
Precompute passages or process them when reading?
The architectural choice involves a trade-off. Preparing representations in advance moves work into writing and maintenance. Computing them during retrieval reduces derived storage but adds work on the path to the response.
The confirmation experiment measured the latter approach. Its 95th-percentile context-building latency was close to the complete-memory baseline, but much higher than the excerpts-only baseline. A claim that it was “no slower” would conceal the choice of reference.
We also evaluated lexical selection among passages that were already available as candidates. In that diagnostic, it identified the expected passage in a majority of cases, with limitations on paraphrased questions. This measures passage selection, rather than the accuracy of a final answer. It cannot settle every architectural choice on its own.
What the experiments taught us
A useful passage is more than a relevant sentence. It needs a continuing connection to its source document, its version and the conditions that restrict its meaning. Retrieval benefits from looking below document level when that finer grain preserves enough context to interpret the answer.
For Nawame, these experiments inform choices about granularity, computation and the controls surrounding retrieval. They connect with our work on treating memories as data and reusing embeddings. The Lab page places that research within the role of durable memory for authorised assistants.
Related reading: Why found does not mean true.


