JUntil recently, the most important question when choosing an AI model was: which one is the best? Which is better at writing code, analyzing documents, solving problems and handling increasingly complex tasks?<\/p>
Today, some companies are starting to ask a different question: what exactly happens to the information we give this model?<\/p>
Palantir doesn\u2019t want to make some of Anthropic\u2019s most advanced models available to its customers without additional guarantees about data retention. Nvidia is restricting the use of Claude for more sensitive tasks. Booz Allen Hamilton has prohibited the use of commercial versions of Claude for some of its cybersecurity work. As Reuters reported<\/mark><\/a>, the common concern is data security and the protection of intellectual property.<\/p> This doesn\u2019t mean there\u2019s suddenly a crisis of confidence in artificial intelligence. But it does highlight a problem that will probably become increasingly important as AI advances.<\/p> Because the more AI can do for a company, the more the company first has to tell it.<\/p> The issue gained prominence after Anthropic changed the rules for its most advanced models. The company introduced a category known as Covered Models. For these models, prompts submitted by business users and the model\u2019s responses are retained for 30 days. The policy also applies to organizations that previously used Zero Data Retention, or ZDR. Under ZDR, the provider doesn\u2019t retain data after processing a request.<\/p> Anthropic explains in its Covered Models documentation<\/a><\/mark> that the change applies to organizations using ZDR directly through Claude as well as through selected cloud platforms.<\/p> For most users, the difference may seem like a technical detail. For a company working with confidential information, however, it\u2019s fundamental.<\/p> If an employee asks a model to polish an ordinary email, 30-day retention probably won\u2019t be a major concern. But the situation looks very different if a developer submits a piece of code from a product under development, an analyst works with nonpublic financial data or a security specialist gives the model details of a vulnerability that hasn\u2019t yet been discovered by others.<\/p> A prompt stops being just a question and it becomes part of the company\u2019s documentation.<\/p> According to Reuters, this is precisely why Nvidia is restricting the use of Anthropic models for more sensitive tasks. Palantir has gone further, seeking a ZDR guarantee from Anthropic that the provider couldn\u2019t later revoke unilaterally. Booz Allen, meanwhile, has banned the use of commercial Claude products for some of its cybersecurity work.<\/p> Companies aren\u2019t abandoning AI. They\u2019re starting to define what AI isn\u2019t allowed to see.<\/p> During the first few years of the generative AI boom, the discussion around data focused primarily on the risk that an employee might thoughtlessly paste part of a contract, customer data or a confidential document into a public chatbot. That problem hasn\u2019t disappeared, but the scale of AI use has changed.<\/p> Models are no longer used only to write text or summarize documents. They\u2019re given source code, product designs, research results, financial data, internal procedures and customer information. Increasingly, they\u2019re also connected to other company systems, while AI agents are given access to databases, applications and code repositories.<\/p> In this setup, the information supplied to the model may be worth far more than its answer. Importantly, storing data isn\u2019t the same as using it to train a model.<\/p> Anthropic states in its policy for commercial products<\/a><\/mark> that inputs and outputs from Claude for Work, the API and Claude Gov aren\u2019t used for model training by default. That can change if the customer explicitly agrees, including by submitting a conversation as feedback. The distinction matters because public discussions often conflate several different issues.<\/p> The first is retention: does the provider keep the prompt and response, and if so, for how long? The second is training: can that data later be used to improve models? The third is access: who can see stored content and under what circumstances? And finally, there\u2019s security monitoring: analyzing how models are used to detect abuse.<\/p> Only when these elements are separated does the real problem become clear.<\/p> The simplest solution would seem obvious: if companies don\u2019t want their data stored, the provider should simply not store it. But from Anthropic\u2019s perspective, that\u2019s precisely where the problem begins.<\/p> The company argues that serious abuse may become apparent only when security systems analyze a sequence of interactions. Its examples include espionage campaigns and attempts to extort data. A single prompt may look harmless. Only a series of requests reveals what the user is actually trying to do.<\/p> That\u2019s why Anthropic requires 30-day retention of prompts and responses for Covered Models.<\/p> According to the company, its employees can\u2019t read these conversations by default. Human access is supposed to occur only through a controlled process, for example when automated security systems flag content as potentially dangerous. Only approved employees can review it, and every instance of access is logged. After 30 days, the data is automatically deleted unless it has been flagged for security reasons or the company is legally required to retain it.<\/p> From a model security perspective, this makes sense. The problem is that from the customer\u2019s security perspective, the exact opposite approach may make just as much sense.<\/p> Imagine a company developing technology worth billions of dollars. Its employees want to use the best available model to analyze code. The AI provider says: I need to retain this data for a while because that\u2019s the only way I can detect serious abuse. The customer replies: precisely because this code is so valuable, I don\u2019t want another copy of it to exist outside an environment I control.<\/p> Both sides are trying to improve security. Just not the same kind of security.<\/p> Anthropic wants to reduce the risk of its models being used for dangerous purposes. The enterprise wants to protect its intellectual property, trade secrets and customer data from leaving its control. Every additional place where information is stored creates another piece of infrastructure that needs to be secured. Even if the provider promises a very high level of protection, some organizations may follow a simple rule: the most sensitive information shouldn\u2019t be stored outside their own environment at all.<\/p> And that\u2019s why ZDR may go from an obscure acronym buried in a technology vendor contract to one of the requirements for deploying AI.<\/p> The paradox is even greater because AI development is moving in the opposite direction.<\/p> Models are supposed to remember context, carry out increasingly long tasks, use multiple tools and keep track of successive stages of a process. That\u2019s exactly what allows them to take on more complex work.<\/p> At the same time, the largest business customers may increasingly expect model providers to remember as little as possible.<\/p> This isn\u2019t just an Anthropic problem. In August, OpenAI announced expanded access to Zero Data Retention for its most advanced models. The company says that for eligible API customers, prompts and responses aren\u2019t retained after a request has been processed, while business customer data isn\u2019t used for training without explicit consent. OpenAI also announced Private Safety Processing<\/a><\/mark>, a mechanism designed to detect dangerous patterns spanning multiple interactions without exposing their content to company employees.<\/p> It\u2019s an interesting direction because it attempts to reconcile two conflicting requirements: analyzing user behavior over time while allowing enterprises to retain control over their data.<\/p>An AI you can\u2019t show everything<\/h4>
The prompt may be worth more than the answer<\/h4>
Why does AI want to remember?<\/h4>
Security versus security<\/h4>
Memory as a problem, not a feature<\/h2>