A call for critical thinking in the age of AI

I was reading this morning about the latest attempt to work out how we govern increasingly powerful artificial intelligence. President Trump and many of the largest AI companies have signed what he described as a “morally binding” agreement. It includes internal controls and external auditing, but it remains voluntary rather than legally enforceable.

My immediate reaction was not especially sophisticated: Oh, shit.

We have spent the past twenty years watching technology companies struggle to police the online systems they already built. Now we are contemplating a similar reliance on self-regulation for artificial intelligence.

That thought sent me down several increasingly meandering tangents: regulation, AI safety, red-teaming, models attacking other models to discover vulnerabilities, and the fascinating idea that some of our best defences against powerful AI may eventually be other AIs specifically instructed to find what everyone else missed.

Somewhere in that wandering thought process, I found myself thinking about John Stuart Mill.

“He who knows only his own side of the case, knows little of that.”

Which led me to a rather different question.

Never mind universal translators. Where is my Ready Room?

For the non-Trekkies, the Ready Room is where the captain can step away from the bridge, gather senior officers, hear different perspectives and then make the decision. It turns out this particular piece of science fiction technology might be surprisingly useful.

People already approximate it with AI. Ask ChatGPT a difficult question. Ask Gemini the same thing. Try Claude. Search elsewhere. Compare the answers and see whether somebody spots something the others missed.

There is something healthy about that. The problem is friction. Every time we move to another system, we potentially abandon something increasingly valuable: context. The next AI may understand the subject, but it doesn’t necessarily understand the project, our previous decisions, our resources or why the question matters.

So give us a button: Ready Room.

A normal AI assistant does something extraordinarily useful. It looks at a complicated field of information, weighs it and gives us a synthesised answer. I remember a similar principle from my time in the army. Some experienced shooters preferred the traditional blade-style sights they had learned to use, while newer circular sights could feel less precise to them. But the newer sight picture made it easier for ordinary shooters to achieve consistent results.

I increasingly think today’s AI assistant works a little like that sight. It centres the important information and gives most of us a pretty good answer, pretty consistently. For everyday questions, that is exactly what I want.

The problem comes when the consequences of being wrong become asymmetric.

If an AI estimates a project at €10,000 and €10,000 really is the most likely outcome, that’s useful. But if there is also a credible scenario where one hidden problem turns it into €40,000, I don’t necessarily want that possibility averaged into a beautifully reasonable €12,000 answer. I want somebody in the room to point at the €40,000 scenario and make its case.

Ready Room mode would let me see some of what the normal answer considered and rejected.

The Advocate could make the strongest case for the proposal. The Sceptic would try to pull it apart. The Analyst would concentrate on evidence, numbers, probabilities and uncertainty. The Pragmatist would ask what those risks actually mean in the circumstances. Finally, the Chair would identify where they agree, where they disagree, and what information might settle the argument.

They shouldn’t manufacture disagreement. Sometimes the evidence really does point overwhelmingly in one direction. If everyone reaches the same conclusion, tell me so. The purpose isn’t five artificial personalities performing a debate. It is to prevent consequential alternatives from disappearing inside a polished synthesis.

Take taxes, because almost everyone understands that particular joy.

A normal AI tax assistant might examine a return and conclude that everything appears correct. In the Ready Room, the Analyst verifies the calculations. The Advocate looks for legitimate deductions that have been missed. The Sceptic challenges whether those deductions actually apply and what evidence supports them. The Pragmatist distinguishes worthwhile savings from aggressive interpretations likely to create more trouble than they are worth.

The Chair might conclude: Your return appears correct. These two additional deductions look well supported. These three depend on assumptions I cannot verify. Check those before filing.

That isn’t AI replacing judgement. It is AI supporting judgement.

The idea becomes more valuable as the stakes rise. For a health question, explicitly ask one perspective to challenge the emerging interpretation. For a legal problem, put the strongest prosecution and defence cases in the same virtual room. For financial planning, let one voice make the investment case while another searches the prospectus for the clauses that could hurt you.

Suppose an investment document says withdrawals normally take six months. The useful question isn’t merely whether the document says six months. The Sceptic should ask: Normally? Under whose control? What exceptions exist? What happens during a liquidity crisis?

Perhaps the answer remains reassuring. Perhaps you still invest. But now you are making the decision having seen the case that didn’t make the headline.

That distinction matters particularly when people have limited resources. A 10 per cent chance that a €10,000 project becomes a €40,000 project means something very different to someone with €500,000 in reserve than to someone with €20,000. The probability hasn’t changed. The consequences have.

There are decisions in my own recent past where I wish I had had something like this. Not because I needed an AI to tell me what to do, nor because the professional advice I received was necessarily unreasonable. I would simply have appreciated being able to put the same information before another room and say:

Make the case I’m not hearing.

We delegate judgement constantly. We rely on doctors, accountants, solicitors, financial advisers, tradespeople and countless other specialists because nobody has the time, money or ability to obtain five independent professional opinions about every decision.

A Ready Room wouldn’t eliminate that dependence. But it could make a second, third and fourth perspective extraordinarily cheap.

There could even be two versions. One Ready Room could know everything the normal assistant knows about us and the project. Another could be deliberately context-blind: hand it the documents and say, You don’t know me. What do you make of this?

A fresh pair of eyes on demand.

Perhaps that is the part of AI development I would most like to see. Not another system promising to make decisions for us, but one deliberately designed to make it harder for us to stop thinking.

Most of the time, give me the reticle. Centre the information, simplify the problem and help me arrive at a sensible answer.

But when the stakes are high, give me another button.

Open the Ready Room.

As always, be excellent to each other.