The problem
What actually happens when someone pastes a client file into a chatbot
Not a hypothetical, and not scaremongering. Just the sequence of events, in order, and where the responsibility lands at the end of it.
- 01
It leaves the building
The moment the text is submitted, it is transmitted out of your network to infrastructure owned by someone else. This is a disclosure. It happened the instant the button was pressed, and nothing after that point can undo it.
- 02
It is processed offshore
Most major AI services process requests in overseas data centres. Which country depends on the provider, the product tier, and how they have chosen to route traffic that day. In practice you will not be told, and you cannot check.
- 03
It is retained — for a while, at least
Providers typically retain conversations for a period, for abuse monitoring and service operation, even where they undertake not to train on the content. Retention policies vary by tier and change over time. A free consumer account almost always has the weakest protections.
- 04
It may be reviewed by a human
Many services reserve the right to have staff or contractors review flagged conversations. That is a reasonable safety measure, and it also means a person you have never met may read a document you were obliged to keep confidential.
- 05
You remain accountable
Under APP 8, disclosing personal information to an overseas recipient generally keeps you responsible for what happens to it. Your client, your regulator and your insurer will look to you — not to the AI provider — and "an employee did it without telling me" is not a defence.
- 06
You cannot prove what was sent
This is the part that causes the most trouble. If you are ever asked to state exactly what confidential material left your firm, you will have no log, no record and no way to reconstruct it. Notifiable breach assessments are very difficult without that.
Be fair about it
The AI providers are not the villains here
The major providers publish their terms, offer business tiers with genuinely stronger commitments on training and retention, and have invested seriously in security. A paid enterprise agreement with a reputable provider is a real improvement over a free consumer account, and for plenty of organisations it is a perfectly sensible answer.
Two things remain true regardless. First, it is still a disclosure to a third party, still usually processed offshore, and still dependent on that provider continuing to honour its terms, stay solvent, and not get breached. Second — and this is the one that actually bites — the staff member drafting an advice at 9pm is not using the enterprise tenancy. They are using the free app on their phone, because it is already logged in.
The gap is not between good providers and bad ones. It is between what your policy says and what your people do.
Why banning it makes things worse
A prohibition does not remove the incentive that created the behaviour — it removes your visibility of it. Staff who were previously mentioning that AI helped with a summary stop mentioning it. The usage continues, on personal devices and personal accounts, entirely outside anything you could audit or manage.
You end up with the worst available combination: the same exposure, none of the oversight, and a documented policy demonstrating you were on notice of the risk. If a disclosure ever has to be assessed, that policy is not the shield people assume it is.
The firms handling this well are not the ones with the strictest rules. They are the ones who gave staff a sanctioned tool that is good enough to actually use, so the unsanctioned one stops being worth the effort.
Which is the whole idea behind OnPrem
Put a capable AI system inside the building, make it free at the point of use, and make it the path of least resistance. Nobody has to be disciplined into compliance, because the convenient option and the safe option are finally the same thing.
How it works →No obligation
Find out whether this fits your firm
Tell us how your team is using AI today and what you cannot afford to have leave the building. We will tell you honestly whether an on-premise system is the right answer — including when it is not.