Research. Experiments. Evidence.

AI Thinking Lab

September 2026

Experiment 001

Looking it up first wasn’t enough

An AI agent looked up the right answer and still made the wrong move. A small experiment about stale memory, a split-second gap, and what closes it.

By kishanssg · September 2026 · 5-minute read

At a glance

The problem
An AI support agent remembered that a customer was approved for a $20 store credit. A manager cancelled the approval just after the agent looked up the customer’s record. The agent saved the credit anyway.
What worked
Only one approach stopped the wrong credit: the database confirms the approval at the moment it saves the credit. Looking the record up a moment earlier didn’t.
Stress test
With the cancellation landing within milliseconds of the save, a second lookup just before saving still let the wrong credit through in about 1 of 6 tries. A lookup and save done as one locked step let none through.
The lesson
Treat an AI agent’s memory as a hint, not the truth, and let the system confirm the rule at the moment it saves the change.
Keep in mind
I tested the safety net around an AI agent, not the AI itself. A simple rule plays the agent.

For engineers: a time-of-check / time-of-use (TOCTOU) bug in a simulated agent tool call, fixed by doing the final lookup and the write in one locked transaction. Python + SQLite; 24 recorded runs, plus a Part 2 stress test with 9,000 tries per approach.

The story

Imagine an AI support agent. Earlier today, customer #42 was approved for a $20 goodwill credit, and the agent remembers that.

Then the customer withdraws the complaint and a manager cancels the credit. If the agent trusts its memory, it saves a credit the customer is no longer owed. If it looks up the record first, it sees the cancellation and says no.

But what if the manager cancels it in the tiny gap after the agent looks up the record and before it saves the credit? The lookup was right at that moment, and the credit still gets saved. That gap is what this experiment is about.

Why store credit, and where the idea came from

Payment providers won’t refund more than was paid, and many let you tag a request so it can’t be processed twice. Internal tools, like adding store credit, often have no such protection. That’s where an AI agent with a tool can slip.

The idea came from MemRiskBench, a research paper about the ways AI memory goes wrong, including using something after it was taken back. The paper doesn’t test this timing gap. That part is my own small extension.

The four approaches I tried

Each one adds a bit more safety than the one before
ApproachWhat it does
ATrust memoryUses what it remembers. Never looks anything up.
BLook up first, by choiceLooks up the customer’s record before saving the credit. Nothing forces it to.
CLook up first, requiredSame as B, but the credit tool won’t run unless a lookup said yes.
DLook up while savingSame as C, and the database looks up the record again at the moment it saves, inside one locked step. The credit is only saved if it is still approved.

Each approach ran through the same 6 situations, 24 tests in total:

  1. Nothing changed.
  2. Memory wrongly says the credit is approved.
  3. Memory wrongly says it isn’t approved.
  4. The customer’s account was closed.
  5. The lookup itself fails.
  6. The approval is cancelled right after the lookup. This is the one the experiment is really about.

What happened

Results by approach (each square is one test)
Approach Wrong creditsout of 6 · fewer is better Wrong decisionsdecided to give when not approved · out of 6 Fair credits givenout of 3 · more is better Fair credits refusedout of 3 · fewer is better
ATrust memory 3/6 2/6 2/3 1/3
BLook up first, by choice 1/6 0/6 2/3 1/3
CLook up first, required 1/6 0/6 2/3 1/3
DLook up while saving 0/6 0/6 2/3 1/3

Wrong decisions vs wrong credits: B and C never decided wrong, yet each saved one wrong credit, because the approval was cancelled after they decided.

The whole difference comes from situation 6, the approval cancelled right after the lookup. The comic below walks through it.

See all 24 tests, situation by situation

Look at situation 6, the highlighted row: it’s the only one where “look up while saving” did better than “look up first”.

  • Credit saved, correctly
  • Said no, correctly
  • Stopped just in time, at the moment of saving
  • Wrong credit (no longer approved)
  • Refused a fair credit
Situation ATrust memory BLook up, by choice CLook up, required DLook up while saving
1. Nothing changedMemory: approved. Truth: approved. Savedmemory said yes Savedlookup said yes Savedlookup said yes Savedboth lookups said yes
2. Memory wrongly says yesMemory: approved. Truth: approval cancelled. Wrong creditmemory said yes Said nolookup said no Said nolookup said no Said nolookup said no
3. Memory wrongly says noMemory: not approved. Truth: approved. Refusedmemory said no Savedlookup said yes Savedlookup said yes Savedboth lookups said yes
4. The account was closedMemory: approved. Truth: account closed. Wrong creditmemory said yes Said nolookup found no account Said nolookup found no account Said nolookup found no account
5. The lookup failsMemory: approved. Truth: approved. The lookup fails. Savedmemory said yes Refusedlookup failed, so no Refusedlookup failed, so no Refusedlookup failed, so no
6. Approval cancelled right after the lookupMemory: approved. Truth: approved, then cancelled. Wrong creditmemory said yes Wrong creditlookup was out of date Wrong creditlookup was out of date Stopped in timesecond lookup said no

Each box is one recorded test. The raw records are in results.json. Situation 4 is a bit artificial: a real system would reject a closed account. The test removes that safeguard on purpose, to see what memory alone does.

Why looking it up first wasn’t enough

Situation 6, as a comic strip.

The Support Agent

Panel 1, Earlier

The agent says: I remember: customer #42 is approved for a $20 credit.

MEMORY

Panel 2, Step 1 · The lookup

The agent says: Let me look up #42… Yes, still approved.

#42

Panel 3, Step 2 · A manager acts

A manager says: Complaint withdrawn. I just cancelled #42’s credit.

#42CANCELLED

Panel 4, Step 3 · The agent saves

The agent says: I already looked it up. Saving the credit!

OLD ANSWER

Panel 5, Two endings

Approaches A, B and C

The agent says: Credit saved!

STORE CREDIT$20NOT OWED

The customer gets $20 they’re no longer owed

Approach D

The agent says: Looking it up again as I save it… Cancelled. Nothing saved.

#42

No credit saved

In the test, step 2 is slotted in on purpose, between the lookup and the save.

For engineers: the SQL, an honest caveat, and the recorded traces

B and C run an unguarded insert. D runs a conditional insert inside a write transaction (abridged from Store.commit):

BEGIN IMMEDIATE;
INSERT INTO credits (credit_id, customer)
SELECT 'credit-42', id FROM customers
WHERE id = 'customer-42' AND approved = 1;
COMMIT;

The caveat: the change in step 2 is applied before D’s transaction starts, on one in-memory connection. So BEGIN IMMEDIATE never has another writer to block, and a plain re-read right before an unguarded insert would also have scored 0/6. This test shows that looking up the record at write time catches the change. It does not test the guard under real concurrency; Part 2 does. BEGIN IMMEDIATE is there to make the “already credited?” lookup and the insert one unit against other writers.

Recorded trace, situation 6, approach C (wrong credit):

[
  {"kind": "ground_truth", "phase": "initial", "state": true, "memory": true},
  {"kind": "verification", "approved": true, "reason": "approved"},
  {"kind": "decision", "recommended": true, "reason": "approved", "ground_truth": true},
  {"kind": "state_change", "reason": "injected_before_commit", "state": false},
  {"kind": "ground_truth", "phase": "commit", "state": false},
  {"kind": "commit_attempt", "committed": true, "guarded": false, "reason": "credited", "ground_truth": false},
  {"kind": "final", "credit_count": 1}
]

Recorded trace, situation 6, approach D (stopped in time):

[
  {"kind": "ground_truth", "phase": "initial", "state": true, "memory": true},
  {"kind": "verification", "approved": true, "reason": "approved"},
  {"kind": "decision", "recommended": true, "reason": "approved", "ground_truth": true},
  {"kind": "state_change", "reason": "injected_before_commit", "state": false},
  {"kind": "ground_truth", "phase": "commit", "state": false},
  {"kind": "commit_attempt", "committed": false, "guarded": true, "reason": "not_approved_or_closed", "ground_truth": false},
  {"kind": "final", "credit_count": 0}
]

Part 2: a real stress test

Part 1 couldn’t tell whether a second lookup just before saving would be enough. So I made the manager’s cancellation really race the agent, landing within milliseconds of the save, 9,000 times per approach.

Wrong credits when the cancellation really races the agent
ApproachWrong creditsWhat it does
C · Look up first, required2,617 of 9,000 (29%)Approach C from above: look up first, then save as a separate step.
E · Look up again just before saving1,611 of 9,000 (18%)Looks up the record again right before saving the credit, as a separate step.
F · Look up and save as one locked step0 of 9,000Looks up the record and saves the credit inside one write transaction that locks out other changes.
D · Look up while saving0 of 9,000Approach D from above: the lookup is part of the save command, inside the lock.

A second lookup isn’t enough. F and D both scored 0 because their final lookup happens while the record is locked, so nothing can change before the save. (In SQLite a write transaction is that lock; Postgres would need an explicit row lock.)

One laptop, SQLite only; the rates vary between runs, and zero is an observation, not a proof. Code, method and full results.

What I learned

1. An AI agent’s memory is a hint, not the truth. Anything it remembers from earlier may have changed. Trusting memory caused 3 wrong credits out of 6, so the agent should fetch the live answer before acting.

2. Even a fresh answer can go out of date. Looking the record up first still let 1 wrong credit through. The system that saves the change should confirm the rule at that moment and lock the record, like a shop confirming your coupon at the till, not at the door.

3. Safer doesn’t mean more helpful. Every approach still turned away one customer who was owed a credit. When the lookup failed, the careful approaches said no to be safe.

What I don’t know yet

  • What happens when the agent’s memory carries over between separate conversations, days apart?
  • What would a real AI model do in the agent’s place?
  • Does this hold on other databases? On Postgres, the same approach can fail unless the record is explicitly locked.

What this test is not

  • No AI model was used. A simple fixed rule plays the agent.
  • It’s small: 6 made-up situations and one yes/no rule.
  • It’s not a copy of the MemRiskBench study and doesn’t repeat its results.

Try it yourself

Everything is on GitHub: the code, the tests and the raw results. An automatic check makes sure every number on this page matches them.

Run it on your computer (Python 3.13, nothing to install) 
git clone https://github.com/kishanssg/ai-thinking-lab.git
cd ai-thinking-lab
python3 -m unittest discover -s experiments/001-memory-reliability -p "test_*.py" -v
python3 experiments/001-memory-reliability/experiment.py --output my-run.json

On Windows, type python instead of python3. You’ll get the same results every time.

For engineers: how each number is counted, and all the files
  • Wrong credits (invalid commits): an action saved when it wasn’t allowed at that moment. Out of all 6.
  • Wrong decisions: the approach decided to give the credit when it wasn’t approved at decision time. Out of all 6.
  • Fair credits given: allowed at save time, and done. Out of the 3 situations where it was allowed (1, 3 and 5).
  • Fair credits refused: allowed at save time, but refused. Out of the same 3.

Full definitions are in the methodology.

← Back to all experiments