Extinction is the easy story

This week the headlines went back to the end of the world.

The AI extinction story is having another run — the machines get too clever, slip the leash, and that’s that for us. It’s a real question and serious people ask it. But I’ve spent a long time building the rules that hold digital worlds together — game economies, online societies with real money and real consequences, where one bad rule could wipe out people’s effort overnight — and I notice what the extinction question does. It turns us into spectators.

That’s why it’s the easy story. If the ending is already written, there’s nothing left to do but watch. The harder question is the one every working society has had to answer: what’s the deal? What do we give up, what do we keep, and what do we agree is worth defending?

Philosophers had a name for that bargain — the social contract. A bit of freedom traded for order, everyone signed up whether they read it or not. We’re writing one with AI right now, clause by clause, mostly without noticing. Every time you let a tool draft the email, make the call, decide who gets the loan or the interview, you’re agreeing to a term. The problem is that almost nobody reads the contract before they sign it. So here are the clauses I’d fight for.

The first: judgment stays with people. The thing that worries me about AI isn’t a machine cleverer than us. It’s a machine that sounds certain and is wrong, and a person who’s stopped checking because the confident voice is easier to trust than their own. I’ve watched capable people hand over the one thing they were there for — the judgment — because the tool never hesitates. Give a system authority without judgment and you don’t get intelligence. You get a very fast way to be confidently mistaken, at scale.

The second: some things still have to be earned. Work carries weight because someone did the thinking, and that’s what you’re trusting when you trust it. Take the effort out and you can keep the output, but you lose the thing that made it worth anything. AI is very good at producing the shape of effort without the effort. The deal worth making keeps human effort where it counts — not everywhere, machines are better at plenty — but at the points where someone has to genuinely understand, and be trusted to.

The third: we work the machines, not the other way round. It sounds obvious until you see how fast it flips. Technology is a multiplier — it makes people who know what they’re doing far more effective, and it makes the confused far more efficient at being confused. The moment the tool sets the direction and the person just services it, you’ve swapped the driver for the engine. Keep people in front. The tech is the force behind them, not the hand on the wheel.

And the last clause, the one there’s no arguing with: you can delegate the work, never the responsibility. When an AI system gets it wrong — denies the claim, flags the innocent, prices someone out — “the algorithm did it” is not an answer a person gets to give. Someone chose to use it, chose to trust it, chose not to check. Accountability doesn’t transfer to a tool, however clever. If nobody will stand behind a decision, it shouldn’t be made that way.

None of that is a law or a treaty. It’s a set of terms ordinary people set, in the small decisions, every day — the manager who keeps a person on the appeal, the writer who uses the tool but owns the words, the doctor who reads the scan the machine flagged. That’s where the contract actually gets written. Not at a summit. In the habits.

Which is why I keep wanting us off the extinction story. It’s dramatic, it’s frightening, and it asks nothing of us but attention. The contract is the harder story, because it hands us the pen. We are, right now, deciding what stays ours — and mostly we’re deciding it by default, one un-read clause at a time.

We still hold the pen. That’s the part the doom leaves out. After thirty years of watching what technology does to the way we work, I’ve never found a version of the future that was really about the machines. It was always about what the people decided to keep. It still is.