Is ChatGPT confidential? What Australian firms need to know

Short version: a conversation with a public AI chat tool is not confidential in the sense a professional obligation requires. It may be private in the ordinary consumer sense — the provider is not publishing it — but that is a much lower bar than the one you are held to.

Here is what actually happens, and where the distinctions matter.

The transmission is the disclosure

The critical moment is not what the provider does with your text. It is the fact that the text left your network at all.

When a staff member pastes a client document into a chat tool, that document is transmitted to infrastructure owned and operated by a third party. For most major services that infrastructure is overseas. Under Australian privacy law that transmission is a disclosure, and it is complete the instant it happens. Nothing the provider does or does not do afterwards changes that.

This is why “but they promise not to train on it” is a weaker reassurance than it sounds. Training is one downstream use among several. The disclosure already occurred.

The tiers are genuinely different

It is unfair to lump every AI service together, so let us be specific about the distinctions that actually exist.

Free consumer accounts typically have the weakest protections. Historically, providers have used free-tier conversations to improve their models by default, with an opt-out buried in settings that most users never find. Retention periods tend to be longer and commitments looser.

Paid individual subscriptions usually sit somewhere in the middle — often better default settings, but still fundamentally a consumer product governed by consumer terms.

Business, team and enterprise tiers are a real step up. These commonly include contractual commitments not to train on your inputs, shorter retention windows, administrative controls, audit logging, and sometimes data residency options. If your organisation has a properly configured enterprise agreement with a reputable provider, you are in a substantially better position than a firm relying on free accounts.

API access is different again, with its own terms that frequently differ from the consumer product of the same brand.

The problem is not that good options do not exist. They do.

The problem is which tier gets used

Every firm we speak with that has an enterprise arrangement also has staff using the free app on their phones.

This is not a discipline failure. It is friction. The enterprise tenancy requires signing in through a work device, on a work account, often through a login flow that has timed out. The free app is already open, already logged in, and sitting on the home screen. At 9pm with a draft due, people use the thing that works in two seconds.

So the firm’s actual exposure is not defined by the best tool it has purchased. It is defined by the easiest tool available to a tired person.

What “we don’t train on your data” does and does not mean

It means the provider undertakes not to use your inputs to improve its models. That is a meaningful commitment and worth having.

It does not mean:

• That nothing is retained. Providers generally hold conversations for some period for abuse detection, security and service operation. Retention varies by tier and changes over time.
• That no human ever sees it. Most services reserve the right to have staff or contractors review flagged content. This is a sensible safety measure that also means a stranger may read a privileged document.
• That the data stays in Australia. Processing location depends on the provider and the tier. Some now offer regional options; many do not, and routing is not something you can independently verify.
• That the terms cannot change. Terms of service are updated. Companies get acquired, restructure, and occasionally fail. Your obligation to your client does not update with them.
• That it cannot be breached. Every organisation holding data is a target. A commitment not to misuse data is not a guarantee nobody else will obtain it.

Where Australian law lands

Under the Privacy Act 1988 (Cth), APP 8 governs cross-border disclosure of personal information. In broad terms, before disclosing personal information to an overseas recipient you must take reasonable steps to ensure that recipient does not breach the Australian Privacy Principles.

The provision that catches most people out is section 16C. Where you disclose personal information overseas, an act by that overseas recipient which would breach the APPs is generally taken to be a breach by you. Accountability follows the data. You cannot outsource it along with the processing.

Alongside that sits APP 11, requiring reasonable steps to protect personal information from unauthorised disclosure, and the Notifiable Data Breaches scheme, which can require notifying both the OAIC and affected individuals where a breach is likely to result in serious harm.

Two practical consequences follow.

First, “an employee did it without telling me” is not a defence. The obligation sits with the organisation.

Second — and this is the one that causes real difficulty — if you ever have to assess whether a notifiable breach occurred, you will need to know what was disclosed. With staff using personal accounts on personal devices, you will have no log, no record, and no way to reconstruct it. Assessing a breach you cannot see is close to impossible.

A note on the small business exemption

Organisations with an annual turnover of $3 million or less are generally exempt from the Privacy Act. Many small firms assume this covers them.

Check carefully, because the exceptions are broad. Health service providers are covered regardless of turnover. So are businesses that trade in personal information, contracted service providers for Commonwealth contracts, and several other categories. A two-person allied health practice carries essentially the same obligations as a hospital.

The exemption has also been under sustained review, and the direction of travel in Australian privacy reform has been consistently toward narrowing it rather than widening it. Building a compliance position on an exemption that may not survive the decade is a fragile plan.

So what is the actual answer?

If someone asks whether they can paste a client document into a public AI tool, the honest response is a question: which tool, which tier, under what agreement, and can you prove afterwards what was sent?

Most firms cannot answer the last part. That is usually the point at which the conversation becomes serious.

The alternative is architectural rather than contractual. If the AI runs on hardware inside your own office, none of the above applies — not because you have negotiated better terms, but because there is no transmission, no third party, and no overseas recipient. The question stops needing an answer.

That is not the right solution for every firm, and we are the first to say so. But it is the only approach that removes the question instead of managing it.