Your AI Arena Prompts Are Probably Training Tomorrow’s Models Without Your Consent

I was running a benchmark suite (a fixed set of prompts used to measure model quality across releases) against three different hosted AI services last weekend and noticed something I should have noticed months ago. Every prompt I typed was being saved on the vendor side and used to retrain future model versions. The vendors were polite about it. The opt-out was buried three settings deep, behind a toggle that resets every time a new model rolls out. The default state is “your prompts are training data.”

This is not a scandal. It is how the industry has settled on operating. The AI arena providers (the sites where you can compare GPT, Claude, Gemini, Llama, and the long tail of smaller hosted models side by side) treat your chat history as a free corpus they can mine. Most of them are upfront about this if you go looking. None of them are upfront about it from the chat window itself. And the dirty secret is that even the “private” or “incognito” modes often only suppress the output from appearing in your sidebar; they do not suppress the training collection on the backend.

What the terms actually say

I went and read the terms of service for the four arenas I use most often. Two of them, the ones most often recommended on Reddit threads, say your prompts may be used to improve their models. One says your prompts will be used unless you turn off a setting inside the chat UI. One says nothing about training at all, which is its own kind of answer. None of them make this clear in the sign-up flow, and none of them put a banner above the text box.

The relevant phrase, when you find it, usually reads something like “we may use your conversations to train and improve our models.” The word “may” is doing heavy lifting there. It is the same word a five-year-old uses when you ask if they ate the cookie. Technically deniable. Operationally, yes.

A few vendors have started adding a “do not train on this chat” toggle. Some of those toggles are per-conversation, some are account-wide, and at least one resets the toggle to off when the model changes underneath you. If you are evaluating a new model release, you have to remember to flip the toggle back on for every new chat. Most people will not.

What the opt-out usually looks like

The standard pattern across the major arenas is:

• A settings page, usually two or three clicks deep from the main UI.
• A checkbox or toggle labeled something like “Help improve [vendor] models” or “Allow my conversations to be used for training.”
• The default state is on.
• The toggle does not retroactively un-train anything you already sent.
• Sometimes there is a separate toggle for “allow human reviewers to read my chats,” which is its own problem.

The retroactivity gap is the one most people miss. If you have been chatting with a model for six months and just now found the toggle, every conversation from those six months is already in the training set. Turning the toggle off now affects future conversations only. A handful of vendors let you submit a deletion request, but the request usually only covers what shows up in your account dashboard. The trained-on copy lives in a different system.

There is also a category gap. Image prompts, voice transcripts, code you pasted in to debug, the prompt you used to generate your kid’s birthday card art, the SQL query you debugged at 2 AM, the draft of a sensitive email you let the model proofread. All of that is the same “conversation” to the vendor. The toggle, when it exists, is all-or-nothing. You cannot say “train on my SQL questions but not my email drafts.”

What “private” mode actually does

Several arenas advertise a private or incognito chat mode. The marketing implies this is end-to-end private. The implementation usually means the chat does not appear in your sidebar history. Some vendors also claim it is excluded from training. A smaller number actually back that claim with an auditable policy.

The trap is that “private” in the chat UI is not the same as “private” in the security sense (a property where only the sender and intended recipient can read the message). The vendor still sees the prompt. The vendor still runs it through their model. The vendor still has it in their logs. What the private mode buys you is the absence of a sidebar entry on your account, which is useful if you share your screen or your laptop, and not useful for anything else.

If you need actual confidentiality for what you are typing, the only path is a self-hosted model. There is no hosted arena that gives you that, because the entire business model of an arena is to collect enough data to either train a model themselves or sell the conversations to someone who is training a model.

How I changed my setup

Three concrete changes I made this week, in order from easiest to hardest:

• Flipped the training opt-out on every arena I use, then verified by sending a test prompt and checking the data export. The export usually shows which conversations are flagged for training. If your test prompt is in the export after you turned the toggle off, the vendor is not honoring the setting. I caught one vendor doing this and stopped using it.
• Moved sensitive work to a local model running on my own hardware. For anything that involves real names, real numbers, real code from a private repo, or anything I would not paste into a public Slack channel, I use a local Llama or Qwen variant (open-weight model families that can be downloaded and run on consumer hardware) on my laptop. It is slower than the hosted frontier models and the answers are not as good, but the prompts never leave the machine.
• Stopped using arenas as my default chat UI for anything I might want to keep private. Arenas are fine for benchmarking, for “what does this model think of this prompt” comparisons, for throwaway creative experiments. They are not fine for daily-driver work. I keep a separate, opt-out-of-training account for benchmarking only, with no real data in it.

The local-model step is the one that took the most setup. A modern quantized model (a version of the model that has been compressed to use less memory, at a small cost in quality) needs about 8 GB of RAM for a 7-billion-parameter size, and runs at maybe 10 tokens per second on a M-series MacBook. That is fast enough for chat. It is not fast enough for the kind of long-context work where the hosted models earn their keep. I use it for short prompts and switch back to a hosted model for anything over about 4,000 tokens of context.

What I would tell past me

Three things, if I could send them back:

• Read the terms of service for any AI tool you use more than twice. Not the privacy policy. The terms. The training clause is usually in the terms, and the privacy policy will sometimes claim the opposite of what the terms say. The terms win in a dispute.
• Default to assuming your prompts are training data until proven otherwise. The vendor has no incentive to make the opt-out easy. The opt-out being hard is a feature for them, not a bug, because every prompt they keep is a small competitive advantage in the next training run.
• Treat arena mode and chat mode as different threat models. The arena is for evaluating the model. The chat is for using the model. Mixing them is how you end up sending your half-finished novel into a training corpus that will surface in someone else’s output six months from now.

Trade-offs

The local-model route is not free. A quantized 7-billion-parameter model on a laptop is roughly 5x slower than a hosted frontier model for the same prompt. The answers are noticeably worse on anything that requires careful reasoning, especially code. The setup cost is real: about 4 GB of download for a quantized Llama, an hour of fiddling with Ollama or LM Studio (two popular local-model runners, both free), and a one-time mental cost of accepting that your local model is the dumber sibling of the hosted version.

The hosted-frontier-with-opt-out route is faster and smarter, but the opt-out is fragile. A model update can reset the toggle. A new feature can introduce a new training surface (image uploads, voice mode, custom GPTs) that the original opt-out did not cover. A vendor acquisition can change the policy overnight. You are trusting the vendor to honor a setting that is hard to verify from the outside.

For most people doing most work, the right setup is a hybrid. Hosted model for the 80% of tasks where the data is not sensitive and the speed matters. Local model for the 20% where the data is real and you would not paste it into Slack. A separate benchmarking-only account for arenas, with no real data, and the training toggle explicitly verified after every model update.

If you only do one thing from this article, go check the training opt-out on the AI tool you used most this week. Right now. Before you finish reading the rest of this paragraph.