Lab

Rewriting a decision can change its status

Short answer

A rewrite can preserve every number and still change a decision. The risk arises when a rejected, tentative or abandoned value becomes applicable. The Nawame Lab investigated this to understand what a meaningful fidelity check must protect.

The same value appears in two texts with different decision status.

The problem was a change in status

Summaries can make memory shorter and easier to retrieve. But a decision is more than a subject and a value. A proposal awaiting discussion and an adopted decision may contain exactly the same amount. Their status determines what a reader may do with them.

Consider a fictional team that considers a new discount and rejects it. A summary retaining the percentage but dropping the rejection creates a commercial rule that was never adopted. The failure is not an invented number. It is a damaged relationship between the value and its status.

Testing actual rewrites as well as constructed errors

A Lab campaign used synthetic notes covering decisions, proposals, exceptions and replacements. It separated deliberately constructed alterations from rewrites actually produced by two models. That distinction proved essential.

Among 151 rewrites of notes containing restrictive values, the audit identified 15 cases that incorrectly made a value applicable, about 10%. The sensitivity analysis matters just as much: nine cases came from one ambiguous French expression. Excluding that expression, the proportion was about 4%. The experiment demonstrates a failure mechanism; it does not estimate its prevalence across business documents.

A promising detector was insufficient

The tested guards combined rules and model-based verification. Some recognised familiar wording but missed unseen expressions. A verifier applied to every pair detected 85% of the constructed alterations. It found only seven of the fifteen alterations in actual rewrites, with about 12% false alerts on that set.

This difference prevents a conclusion based on a single detection rate. A detector may recognise errors constructed around the benchmark’s assumptions while missing errors that writers actually produce. Using rules to reduce verification calls can also exclude precisely the cases that require closer reading.

Ambiguity should remain visible: scope and limits

An ambiguous sentence calls for clarification or a cautious account. Turning it into a clean assertion removes the warning that should have slowed the reader down. A distant exception or a decision that never took effect presents a similar risk.

Manual review covered 79 pairs flagged by at least one benchmark mechanism; unreviewed pairs counted as negative. This could leave alterations undetected. The audit had limitations of its own: short French texts, two writing models, one repetition and a review that was not blind. An exploratory stronger-model approach suggested a possible research direction, but did not provide independent confirmation. It raised another question rather than establishing a reliable safeguard.

The consequence: preserve the original relationship

The experiment strengthens the case for keeping source text, decision status and version history. A reformulation must remain identifiable as a transformation. If a decision is uncertain, the original gives readers a way to challenge the interpretation instead of inheriting manufactured certainty.

For Nawame, this directs research towards the fidelity of relationships between sources, versions and representations. It does not justify routinely passing private memories through an internal language model. Reasons and exceptions and provenance are part of the memory itself, rather than optional notes attached to a shorter summary.

Related reading: How to Store an Inference Without Turning It Into a Fact and How AI can retrieve your past trade-offs without deciding for you.

Your next prompt should not have to contain your whole story.

Start building a memory your AI tools can find again when it matters.

Start free