Aim to be forgotten

Arthur C. Clarke famously said, “Any sufficiently advanced technology is indistinguishable from magic.”¹ I think good engineering eventually disappears beyond that. Why? Magic asks to be noticed. Good engineering becomes ordinary. Then obvious. Then forgotten.

The best engineering often ends this way. An architecture that does not merely impose form, but somehow preserves the full life of the thing: how it is used, perceived, changed, and inhabited. A migration that lands, and nobody can quite say when the old system became the new one. A service stops paging the on-call so quietly that, six months later, people have forgotten it ever did. A boundary is redrawn, and a whole class of bugs loses the conditions it needed to exist. And we could go on.

Nothing happens. That is the point.

And nothing happening may be the hardest engineering outcome to account for.

The evidence problem

Failure is unusually good at producing evidence.

An outage leaves alerts, timelines, tickets, postmortems, and a room full of people being visibly useful. Somebody restores production at two in the morning, and nobody needs convincing that difficult engineering happened.

Prevention leaves almost nothing.

The incident that never happened has no timestamp. The customer who never encountered the bug never opened a ticket. A migration that preserved every contract produces no dramatic before and after.

A system becoming uneventful looks remarkably similar to a system that was always uneventful.

Incidents manufacture evidence. Prevention destroys it.

That creates a strange accounting problem inside engineering organizations. We can count outages. We can measure recovery time. We can reward the save.

It is harder to count the 2 a.m. 911 that never happened.

This does not make incident response less important. Systems fail, including well-built ones. The mistake is confusing the visibility of the work with its value.

A culture that can only see the fire will eventually overvalue firefighting.

The quality without a name

Christopher Alexander spent much of The Timeless Way of Building describing a quality he believed the best buildings and places possessed.²

He called it the quality without a name.

Alive was close. Whole was close. Comfortable, exact, free, egoless, eternal. None of the words was sufficient on its own.

The important part was that the quality could not simply be applied afterward. It emerged when the parts belonged properly to one another and to the life happening around them.

The important part was that this quality was not something you could design into a finished thing by adding the right features or refinements. It emerged when the parts belonged properly to one another, and the whole belonged to the life taking place within and around it.

I think software has something similar.

We call good systems reliable, simple, maintainable, resilient, elegant, boring.

None of those is quite enough.

Sometimes a system is simply right.

The boundary sits where the responsibility actually changes. The data has one place where it means what everyone thinks it means. Failure happens where it can be understood. The common path is easy. The dangerous path is difficult to express. A change that should be local remains local.

Nothing about this announces itself as exceptional. It just works.

The thing we are reaching for may be closer to fit than elegance.

The abstractions fit the domain. The operational model fits the failure modes. The software fits how people actually use it.

When those relationships are wrong, we feel them everywhere: exceptions, coordination, tribal knowledge, documentation explaining why the obvious thing cannot be done.

When they are right, much of that disappears.

The architecture stops announcing itself as architecture. Its presence is felt through the absence of friction.

Boring is made

Migrations and reliability work taught me this better than almost anything else.

A good migration is almost offensively boring.

Compatibility is preserved. Changes are staged. Failure has somewhere safe to go. The system remains observable while it changes. There is a way back.

There may be months of engineering underneath it.

The user experiences Tuesday becoming Wednesday.

Reliability works the same way.

The goal is not to become exceptionally good at panic. It is to make panic increasingly unnecessary.

Make an operation safe to repeat.

Turn something everybody has to remember into something the system guarantees.

Move an implicit assumption into an explicit boundary.

Make the wrong thing harder to write.

None of these is reliability by itself. They create the conditions from which reliability emerges.

Do this long enough and the character of a system changes.

People stop having conversations they once had every week.

A runbook gathers dust.

Someone joins the team and is surprised to learn that a subsystem used to be temperamental.

The old problem starts to sound historical.

Eventually, the current behavior feels obvious.

That is when the work has almost completely disappeared.

From magic to “of course”

I find “of course” more interesting than magic.

Of course the payment went through once.

Of course deploying this service did not take the site down.

Of course these systems agree about who the user is.

Of course I can retry this operation.

“Of course’ means somebody no longer has to carry the mechanism in their head.

That is abstraction at its best: permission to stop knowing something.

The complexity has not vanished. It has been resolved into a boundary trustworthy enough that someone else can forget it.

That may be the closest software gets to Alexander’s quality without a name.

Not timeless because the implementation lasts forever.

Timeless because the thing becomes sufficiently fitted to its world that it stops feeling like a negotiation with complexity.

It simply belongs there.

Make the invisible legible

There is one problem with disappearing too well.

For a long time I thought good work should speak for itself. To explain what it had prevented felt almost self-defeating: asking people to notice work whose success was precisely that there was nothing to notice.

I no longer think that is a virtue.

If good engineering destroys its own evidence, somebody has to preserve the causal story.

Not to manufacture importance. To describe what actually happened.

This boundary removed these failure modes.

This migration retired these operational risks.

This work turned something people had to remember into something the system now guarantees.

This quarter was boring because somebody made boring possible.

Good engineering should become invisible to the people who depend on it.

Its value should not become invisible to the people responsible for sustaining it.

So aim to be forgotten.

Build systems that do not need your memory to remain correct. Remove the special knowledge. Eliminate the recurring intervention. Make yesterday’s difficult problem become tomorrow’s unremarkable assumption.

Let the user forget there was ever a problem.

Let the system forget it ever needed you.

But leave enough understanding behind that the organization remembers what made forgetting safe.