Talking to an assistant stopped being a novelty about a year ago. People dictate a task while walking to the station, dictate a reply in the kitchen with the extractor fan on, dictate a half-formed thought at eleven at night because typing it out is too much work. It reads as the most harmless thing in the toolkit. You are not uploading a document. You are just talking.
What leaves the device is audio.
That distinction is the whole essay. If the transcription happens in the cloud, the recording of your voice goes to a company, gets turned into text there, and the retention of the audio is governed by a policy you have not read and a toggle you have probably not found. Do that twice a day for a year and a corpus exists: you, speaking, in every state you were in.
The usual framing is wrong, and it makes people careless
The version of this risk that gets written about is voice cloning for fraud, and the cases are real. A finance team authorised twenty five million dollars on a video call with a synthetic executive. In January a European financial firm lost thirty five million to a real time clone of its CFO on a Teams call. Cloning a voice from public audio costs under fifty dollars.
And that framing produces a shrug from exactly the people who should care. Any listed company’s CFO has hours of public audio: earnings calls, conference panels, recruitment videos. If a clone only needs a voice, the attacker already has everything, and nothing you do with your assistant changes that.
The shrug is wrong, because public audio is performed audio. It is prepared, rehearsed, delivered into a microphone in a quiet room, and then edited by someone whose job is to make it sound composed. A model trained on that produces a person giving a presentation.
Nobody commits fraud by giving a presentation.
What the corpus adds
The call that works sounds like this: sorry, I’m on the street, can’t talk long, we need one point five to the Hong Kong account in the next thirty minutes, I’ll explain later. Short, pressured, slightly out of breath, traffic behind it. The recipient does not compare spectrograms. They check whether the person sounds like themselves in a hurry.
Four things make that possible, and none of them are in a conference recording.
The unguarded register. How someone speaks when they are not performing: the fillers, the self-interruptions, the sentences abandoned halfway, the way they say a colleague’s name when they are annoyed with them.
Effort and breath. Dictation happens on the move. The corpus contains the same person walking uphill, catching their breath, speaking through a yawn. That is precisely the acoustic state an urgent call needs, and it is the one state that never appears in polished material.
The same voice across conditions. Kitchen, car, street, late night. A model that has only seen the studio has to invent everything else. A model that has heard a year of ordinary life does not have to invent anything.
The pretext. The hardest part of a fraudulent call is not the voice, it is knowing enough to be plausible: which project is late, who reports to whom, what the internal shorthand for the quarterly forecast is. Dictation is full of that, because the whole point of dictating is to move work forward.
Put those together and the corpus is not a voiceprint. It is a behavioural print, and the voice is merely how it is delivered.
The channel nobody counts as a system
There is an assumption worth making explicit, because it inverts the usual threat model.
A negotiation on a call is usually transient. Unless someone turns recording on, the audio passes through and is gone. Meetings feel sensitive, so they are treated as sensitive: recording is a decision, the recording has an owner, retention has a rule.
Voice input to an assistant is the opposite. It is not felt as a system at all, it has no owner inside the company, it is nobody’s line item, and by design the audio must leave the device to be useful. The soft target is not the boardroom. It is the tool that never appeared on the register of systems because it looks like a keyboard.
And beneath the large providers, where at least there are policies to read and enterprise tiers to buy, sits a tier with no scrutiny at all: the AI companion, the language tutor you practise speaking with, the talking toy. An hour a day of unguarded speech about your life, collected by a small company that is itself routing the audio through somebody else’s recognition service. The chain has four parties before anyone has done anything wrong. Transparency obligations for synthetic voice are arriving, and general data protection law has applied to biometrics all along. Enforcement against a mid-sized app is theoretical.
What to ask for
The useful move is not warning people. It is giving them four questions with binary answers, because a security officer can act on those and cannot act on a mood.
Where does recognition happen? On the device or inside our tenant is an answer. In the vendor’s cloud is a different answer, and it should be a decision rather than a default.
Is the audio retained after transcription, and for how long? Zero retention for audio exists as a product. If it is not switched on, it was not switched on for a reason nobody can name.
Is voice excluded from training, in writing? Not a toggle in a consumer app. A clause.
Who can pull the archive, and is that access logged? A corpus nobody can list is a corpus nobody can protect.
Then one control that does not depend on any vendor and does not care how good the clone is. Any instruction to move money, change bank details or bypass a process gets verified out of band: hang up, call the number you already have, or use a phrase agreed in advance. It is old-fashioned, it costs nothing, and it is the only layer that survives a perfect imitation. The finance teams that lost tens of millions did not lack technology. They lacked a callback.
What I do, and what it does not solve
My own dictation runs through a local model. The audio never leaves the machine; what leaves is text I have already seen. That is not virtue, it is architecture: I built the bridge that way because I wanted transcripts I could keep, and the privacy came along as a side effect.
It is worth being exact about what that buys. It protects the recording, not the conversation: the words still travel to whichever model I am working with, and if the content is sensitive then local transcription has done nothing about it. It also moves the risk rather than removing it. A year of my speech now lives on a disk in my flat, which is safe from a vendor’s breach and extremely unsafe if the laptop walks off. Local storage is a decision about who holds the corpus, and it comes with the obligation to decide how long to keep it and whether it is encrypted at rest. I would rather make that decision than discover it was made for me by a settings page.
None of this is a reason to stop dictating. It is faster, it changes how I think about a problem, and I am not going back to typing tasks out. It is a reason to know that the harmless little habit produces an asset, that the asset has a location, and that somebody should be able to say who can read it.
Right now, in most companies, nobody can.