All notes
The AI just started profiling you in the background

June 5, 2026

The AI just started profiling you in the background

Until this week, ChatGPT only remembered what you told it to. As of June 4 it 'dreams' — a background process reads across all your past chats and quietly builds a model of you, keeping it current on its own. That's a genuinely useful upgrade and the moment a chatbot became a profiler. The EU's data regulator said exactly that, today. Here's what actually changed, in plain terms — and why it's the grounding problem aimed at you.

Until this week, ChatGPT's memory was simple and honest: it remembered the things you explicitly told it to remember, and nothing else. On June 4, 2026, that changed. OpenAI started rolling out a new memory system it calls "Dreaming V3" — and the short version is that the model now studies your past conversations in the background and quietly builds a profile of you, without being asked.

It's a real upgrade. It's also a bigger shift than the word "feature" suggests, and it's worth understanding in plain terms.

What actually changed

The old memory was a list. You'd say "remember that I'm vegetarian," and it wrote that down. You could read the list and delete from it. Boring, but you knew exactly what it knew.

Dreaming is different in kind, not degree. A background process now reads across many conversations and synthesizes what it thinks is true about you — automatically, continuously, without a "remember this". It even keeps itself current: OpenAI's own example is a memory that reads "you're going to Singapore in July" rewriting itself to "you went to Singapore in July 2026" after the trip. And because a recent change cut the compute about 5×, this is heading to everyone, not just paying users.

So the model is no longer remembering what you said. It's inferring what's true about you and storing it.

This is profiling — and the regulator said so today

That sounds like a strong word. It isn't mine. Today, June 5, 2026, the European Data Protection Board issued a preliminary opinion stating that persistent AI memory constitutes profiling under GDPR, which triggers obligations around consent and the right to be forgotten. The EU AI Act's transparency rules for chatbots land on August 2 — weeks after this rollout.

And the scale of "automatic" is the part to sit with. A study of 2,050 memory entries from 80 users found that 96% were created by the system on its own, not by anyone asking it to remember anything — 28% of them were personal data in the GDPR sense, and 52% were psychological inferences about the user. The machine isn't transcribing what you told it. It's drawing conclusions about who you are.

It's the grounding problem, pointed at you

I've written a lot about grounding: an LLM is a guesser that produces fluent, confident, sometimes-wrong "facts," so you must never let it author the facts — you ground it on a source of truth and let it only rephrase. Dreaming hands the guesser the one job I keep saying it must never have: authoring facts. Except now the facts are about you — your preferences, your constraints, your psychology — inferred, not stated, and saved.

And here's the sharp edge: reporting notes the new system limits the audit trail — the synthesis is opaquer than the old list. So a model that's structurally capable of being confidently wrong is now confidently deciding what's true about you, and you can't fully see or trace how. That's the memory-as-attack-surface problem and the grounding problem at the same time, aimed at your identity instead of a chatbot's answer.

The honest other side

I don't want to doom this, because it's genuinely good UX. Continuity is the thing chatbots always lacked — an assistant that forgets you every morning isn't much of an assistant. And OpenAI did ship real controls: a readable summary page of what it's synthesized about you, the ability to edit or delete entries, and temporary chats that store nothing. That's the right direction.

The problem isn't memory. It's that the memory is now auto-authored and harder to inspect, and it's opt-out, not opt-in. The default flipped from "it knows what you told it" to "it knows what it decided," and you have to go looking to find out which.

What it means if you build

If you build products on these models, absorb this: "the AI remembers the user" is becoming a default, and the moment it does, you are running a profiling system whether you meant to or not. That's not a UX detail anymore; it's consent, right-to-erasure, and being able to show a user a truthful, complete account of what's stored about them. Treat the user's memory like the regulated, user-owned data it legally now is — inspectable in full, deletable on demand, grounded in what they actually said — not a hidden convenience that quietly accumulates conclusions.

The upgrade everyone is about to love is also the moment the chatbot became a profiler. Useful, yes. But know what changed: it's no longer remembering what you told it — it's deciding what's true about you, and keeping most of that to itself. The least you should demand of any AI that remembers you is to see, in full, exactly what it thinks it knows — and to make it forget.

Comments

No comments yet

Sign in to join the conversation.

Be the first to share a thought.