Every AI vendor will hand you a SOC 2 report. Almost none of them can tell you what was actually in the context window.
The report is real. It is also answering a different question than the one your auditor is going to ask.
A SOC 2 describes the vendor's controls. It says nothing about what your data did on one specific call. In a regulated shop the question is narrower and much harder: what left the building, when, and can you show it afterward.
An agent makes that harder than a chatbot did. It loops. Because every pass can pull in another file or record or query result and all of it rides along in the next call the payload going out is not something anybody designed, it is whatever the loop happened to touch by the time it finished. Nobody planned that. It accumulated.
"We do not train on your data" is a good policy and a bad answer. Training was never the only thing that happens to data that leaves.
So the question I would ask before signing: show me, for one specific run, exactly what text went to the provider. Not the policy. The payload. If a vendor cannot produce that, they are not hiding it from you, they do not know either.
I build the tool layer so the call runs as a program and only the result comes back into context. That started as a cost decision and cuts token consumption 85 to 99%. The part that matters in a regulated environment is the side effect: a payload small enough to enumerate is a payload you can put in front of an auditor.
Healthcare payer integrations, PHI-aware edge architecture, secure file transfer at enterprise scale. I have spent thirty years designing around what data is not allowed to do. The model is new. The constraint is not.
If you are buying AI work anywhere with an audit requirement, ask to see the payload for one run.