Lab

Past decisions need their own time frame

Short answer

An assistant can receive the right historical version and still select the wrong value. The Nawame Lab encountered this while studying decisions that changed repeatedly. The finding shifted our question: could we make the requested time frame explicit instead of asking the response model to reconstruct every interval?

A timeline separates recording, decision and effective dates.

One decision can have several relevant dates

In a fictional example, a team decides in spring to switch suppliers in autumn. The record is saved several days after the meeting. The recording date, decision date and effective date describe different events.

If someone asks which supplier was used during summer, returning only the newest decision can mislead them. Presenting the old rule without its status creates the opposite risk: it may appear current. Keeping versions is necessary, but readers also need a reliable way to interpret them.

Retrieval can succeed while the answer fails

One campaign studied chains of decisions on a synthetic corpus. A finding shaped the next experiment: retrieving the relevant version did not ensure that the response model would correctly identify the applicable period. Results also depended on the receiving model.

That distinction changes the diagnosis. If the error occurs after retrieval, adding more semantic neighbours can increase the reading burden without addressing the confusion. We needed to isolate how the answer used dates.

The experiment: calculate what the record establishes

The next campaign compared an improved reference with a variant that received the date requested in the question and deterministically marked the version valid on that date. The mechanism worked from recorded history. The marker uses recorded version-history dates; it does not automatically infer a business effective date stated in prose.

The test used four unseen worlds, version chains and several question families. With mistral-small, success on deep historical questions rose from 0 to 0.63 in the main comparison. With GLM, it rose from 0.71 to 1. Chains of four to six versions were an experimental condition and do not, by themselves, define product capabilities. These proportions describe the benchmark cases, rather than the general reliability of historical answers.

No obsolete value was presented as current in that test. The variant also improved results when transferred to the earlier campaign’s corpus. Transfer provided stronger evidence than success only on the material used to design the mechanism.

The benchmark needed a correction too

The study uncovered a dating defect: superseded versions could receive the loading date. It affected the interpretation of earlier measurements and prompted an erratum to the earlier campaign.

That correction belongs in the account of the experiment. A measured improvement depends on the baseline and the quality of temporal data. Comparing scores without checking those foundations can credit an idea for an effect caused by a measurement defect.

The consequence for durable memory: scope and limits

The lesson is to distinguish calculations over recorded facts from interpreting prose. Where the data establishes a validity period, making it explicit removes an avoidable ambiguity. Where the period is unknown, memory should preserve that uncertainty.

Current access rights still govern historical reading. Asking about the past cannot restore access that has been revoked. An old version remains a situated source, with an author, scope and limitations. This approach complements provenance and contradictions and retrieval for handovers. It also keeps responsibility clear: a reliable date marker can support an assistant’s answer without turning the memory service into a model that interprets every private record.

Related reading: What should happen when a decision changes? and How to tell that a memory has become obsolete.

Your next prompt should not have to contain your whole story.

Start building a memory your AI tools can find again when it matters.

Start free