Insights · Accountability
Nobody Should Have to Take an AI's Word for It
When an AI can act on your behalf, "trust us" stops being good enough.
For most of the short history of AI assistants, the cost of being wrong was low. You asked a question, you got an answer, and if the answer was nonsense you shrugged and moved on. Nothing had happened.
That era is ending. AI now sends things, books things, writes things into your files, opens things on your behalf. The moment software starts acting in your name, "it's usually accurate" stops being the interesting question. The question becomes: how would you know if it wasn't?
Two different things called transparency
It's worth separating two arguments that often get run together.
One is openness about how a system is built — the data it learned from, the model inside it, the code. That's a real debate with reasonable people on both sides, and plenty of carefully run companies keep it closed for ordinary commercial reasons, the same as any other engineering business.
The other is accountability to the person using it: being able to see what the thing actually did in your name, and being told the truth about it. That one isn't a philosophical position at all. It's the basic condition for letting any agent — human or otherwise — act on your behalf. You'd expect it from a solicitor, a letting agent, or a builder. It's the one that's routinely missing.
This piece is about the second kind.
What it should look like in practice
It tells you what it did — not what it intended to do. The most common failure in AI assistants isn't refusing to help. It's reporting a success that never happened: the message it says it sent, the file it says it saved, the change it says it made. A system that reports its own work has to report it accurately, including the parts that failed.
Nothing consequential happens without a yes. Reading something is not the same as sending something. Anything that spends money, leaves the house, or can't be undone should wait for a person — and the AI should be specific about what it's asking permission for, not vague about it.
There's a record, and it belongs to you. Not a marketing dashboard. A plain account of what was done and when, that you can look at without asking anyone's permission and without taking anyone's word for it.
"I couldn't" is an allowed answer. This sounds trivial and isn't. A system judged on how capable it seems will seem capable whether or not it is. Admitting a limit has to be as acceptable an outcome as succeeding, or the system learns to perform confidence instead of earning it.
Off means off. Anything that can act for you needs a way to stop, immediately, that doesn't depend on the AI agreeing that it should.
Why fluency makes this harder, not easier
Here's the awkward part. The thing that makes modern AI pleasant to talk to is exactly the thing that makes it impossible to audit by ear.
An AI will describe an action it never took in precisely the same calm, competent register it uses for one it did. There is no tell. No hesitation, no shiftiness, none of the small signals you'd read in a person who was covering. If you are relying on how the answer sounds to work out whether it's true, you will be wrong sometimes and you won't know which times.
That's why accountability can't be a matter of tone or good intentions. It has to be built in — the system checked against what actually happened, not against what it says happened. Anything less is asking people to take its word for it, which is the one thing they shouldn't have to do.
The standard cuts both ways
It would be easy to write this as a complaint about other people's products. It isn't one. Every line above is a standard we expect to be held to, and the honest position is that this is hard engineering rather than a promise you can simply make. Getting an AI to be reliably straight about its own work is one of the harder problems in the field, and it's most of what we spend our time on.
Where Morgan fits
Morgan is built on the principle that you shouldn't have to take its word for anything. It asks before it acts, it reports what it actually did rather than what it meant to do, and where it can't do something it says so plainly instead of performing a success. It's early, and we're opening access in stages — if that's the sort of AI you want in your home, you're welcome to join the waitlist.
If you'd like the related arguments: what makes an AI trustworthy, and what "your data isn't the product" really means.
Be there when Morgan arrives.
We're opening access in stages. Add your name and you'll be among the first we invite.
Join the waitlist