Append-only event logs: never edit, always correct

A retail chain discovers one day that three hundred members in its loyalty database now share the same birthday. The questions start immediately. Who changed it? When? Was it a person, or an AI assistant tidying the data? What were the values before?

If the system only keeps the current data, those questions have almost no answers. The data is what it is now, and how it got that way is anyone's guess. At best someone digs through a chat history or asks whoever was on duty whether they remember anything.

As more work is handed to AI, questions like this come up more often. An AI is fast; it can change a thousand records overnight, and if there is a small mistake, it spreads that mistake just as fast. What we need is not a more careful AI but a record that cannot lie: who did each thing, when, with what permission, and what happened — and a record that cannot itself be altered.

This article explains how we designed that record. Its core rule fits in one phrase: append only.

What append-only means

An ordinary table lets you add, change and delete rows. An event log allows exactly one action: add a new event at the end. Events already written cannot be changed or removed.

An event is something that has already happened. "A list was created." "A hundred and twenty people were added." "Sending was scheduled for Friday at 8 p.m." "Sending finished." Each sentence is one event, and they line up in the order they happened, like a ledger you can only write forward in.

The rule sounds strict, even awkward. What if something is simply wrong? But that awkwardness is exactly what makes the record trustworthy. With a record that can be edited, you never know whether it has been. With a record that can only grow, what you see is the order in which things happened, nothing added and nothing missing.

Append-only has another, often overlooked benefit: several writers can work at once without overwriting each other. If two AI workers both edit "the current data" for the same list, the second silently wipes out the first. If both only append events, both actions are kept in a clear order, and a conflict becomes something visible that can be dealt with rather than something that quietly disappears.

Mistakes are corrected, not edited

People make mistakes, and so do AI systems. Append-only does not pretend errors do not exist; it handles them differently.

Say the third event recorded the send time as Friday at 8 a.m. when it should have been 8 p.m. We do not go back and change that entry. We append a new one: "Correction to entry 3: send time should be Friday 8 p.m.; a.m. and p.m. were swapped on input." The correction explicitly points to the entry it corrects.

This has several benefits. The error itself is kept, so we can later work out why it happened — an unclear form, or a gap in some AI's rules. Who noticed the mistake, when, and how it was fixed is recorded too. And any person or program reading the log knows to use the corrected value whenever it meets a correction, with no extra judgment needed.

Accountants have worked this way for a long time. When a ledger entry is wrong, they do not paint over it. They post a reversing entry to cancel it and then a correct one. Both are visible, and the books are right.

A correction is an ordinary event in its own right, recording who made it, when and under what permission. If a correction is itself wrong, another correction is appended. That sounds fussy, but it rarely happens, and every layer stays traceable, which is far more reassuring than data that has been changed several times with no trace.

What each entry records

An entry is more than "what happened". To answer questions later, every entry includes at least:

• Who: the person, or the AI and the person it is acting for.
• When: an exact time, not "this afternoon".
• Under what permission: which authorization or decision card allowed it.
• What: the kind of action and its target, such as "added 120 people to the member list".
• Result: succeeded, failed, or unknown.

The one most often left out is permission. Many logs say who did what but not why they were allowed to. When something goes wrong, nobody can tell "done under the rules" from "done without approval". Recording the permission turns a running diary into evidence that can be checked.

Rejected requests are recorded too. If an AI tries to change member data and is blocked by permissions, that is an event as well. A sudden rise in blocked attempts is often the first sign that something is wrong somewhere.

Where we can, we name these fields the way the wider industry does rather than inventing our own, so the log can later be connected to other analysis tools without translation.

With these fields in place, many everyday questions can be answered from the log instead of by asking around: who last changed this list, which outside actions from last week still have no result, how many times a given AI was blocked this month. The more consistent the entries, the more these become a simple query.

Each entry carries the previous fingerprint

Append-only is a rule, and rules can be broken. If someone opens the file and quietly changes an entry in the middle, how would anyone know?

Our answer is that every entry carries the fingerprint of the one before it. A fingerprint is a short string computed from the content; change a single character and the fingerprint is completely different. Entry n stores its own content plus the fingerprint of entry n−1, and then its own fingerprint is computed and passed on to entry n+1.

Each entry is now locked in by the one after it. Change entry five and its fingerprint changes, so the old fingerprint stored in entry six no longer matches. To hide that, you would have to change entry six, then seven, all the way to the end. A check that walks the chain regularly, or a copy of the latest fingerprint kept somewhere else, is enough to catch any tampering.

None of this needs special equipment or new technology; version control tools and ledger systems use the same idea. Its value is that whether the log can be trusted no longer depends on everyone behaving well. It can be checked by a program.

Replaying events to rebuild status

The log is not only for investigations. It has an everyday job: rebuilding the current status.

Because everything is recorded as an event, you can start from the first entry and apply each one in order to get the state at any point in time — how far a task has got, who holds it now, which outside actions are still waiting for a result. Corrections are applied as they appear. This is called replay.

Replay brings two things. Status can never drift from the record, because status is computed from it; there is no separate status table to keep in sync. And when a new AI or a new colleague takes over, they do not need to ask where things are. They read the log, replay it, and know.

Back to the three hundred birthdays. With a log like this, the investigation takes minutes: find the events that changed the birthday field, see who made them, when and under what permission, and read the values before. Fixing it means appending a batch of corrections with a stated reason.

Replay also makes what-if questions easy. To see what things would look like if one event had not happened, skip it during replay. To see the state of a task at the end of last month, replay up to that day. Questions that are hard in a system that only keeps current data become a matter of choosing where to stop.

Sensitive content: IDs and fingerprints only

The more complete a log is, the more likely it is to capture things it should not. Contract terms, members' contact details, internal pricing — if they are written into the log verbatim, the log itself becomes something that must be tightly guarded, and since it is append-only, they cannot be removed later.

So what goes into an entry depends on how sensitive the data is. Public or internal material can be recorded directly. Confidential material is recorded only as an ID and a fingerprint, never the text itself. The most sensitive material does not enter the event log at all.

An ID and a fingerprint still answer most questions: which document was touched, by whom and when, and whether its content still matches what was approved. The content itself stays where access is controlled and can be retrieved with the right permission.

This has to be decided on day one, not patched in after something goes wrong. Because the log is append-only, confidential text written into it by accident is hard to clean up. So the program that writes to the log checks the data level itself and blocks anything that does not belong before it is written, instead of relying on every writer to remember the rule.

Being honest about what the log cannot see

One more thing worth saying plainly: a log only records actions that pass through it. Every action taken through the shared entry point leaves an event automatically. If a person or an AI goes around that entry point and edits files directly, the log does not see it.

We do not pretend this gap is not there. We list it openly, run regular checks that compare the actual files with the state rebuilt from the log and raise a flag when they differ, and keep bringing more actions through the shared entry point. A record is valuable because what it says is true — including being clear about what it did not see.

A log can survive mistakes. What it cannot survive is quiet edits.

In short

• Event logs are append-only; mistakes are fixed by a correction entry that points to the original and states why.
• Every entry records who, when, under what permission, what and the result, including requests that were refused.
• Each entry carries the previous entry's fingerprint, so tampering anywhere in the chain can be detected.
• Status is rebuilt by replaying events, so newcomers read the log to know where things stand; confidential content is kept to an ID and a fingerprint.