First, employees start using a public chatbot. Then the company buys team access, connects its own documents or adds a customer support tool. Only at one of the later stages do these questions come up: What data goes into the system? Is the company allowed to process it that way?<\/p>
This isn\u2019t just a hypothetical scenario. The Personal Data Protection Office says outright<\/mark><\/a> that the question of legal compliance usually comes up when an AI tool is already running and the decisions about the provider and the data being used have already been made. That means legal analysis doesn\u2019t precede deployment. It\u2019s trying to catch up.<\/p> The scale of the problem is illustrated by a survey conducted by UODO<\/mark><\/a> among 492 organizations. Only 16.7% of respondents reported using AI in their day-to-day operations. Others were running tests, planning deployments or were interested in the technology but didn’t know where to start.<\/p> At the same time, 38.2% of respondents said that their use of AI systems doesn’t involve processing personal data. Another 20.3% couldn’t assess it. That adds up to 58.5% of organizations that either didn’t see a link between using AI and personal data or weren’t sure such a link exists.<\/p> The responses about preparedness for implementing the GDPR are even more interesting. Only 4.1% of organizations considered themselves very well prepared. Another 25% chose “somewhat prepared,” while 44.9% couldn’t assess their own readiness. The remaining respondents judged their organizations as somewhat or completely unprepared.<\/p> UODO summed up these results by saying that 95.9% of organizations are unprepared or unsure of their readiness. That\u2019s a broad interpretation, because it also includes organizations that described themselves as somewhat prepared. Far more telling, though, are two other figures: only 4.1% of respondents were confident they were well prepared, and nearly half couldn\u2019t assess it at all.<\/p> We should be cautious here. The study wasn\u2019t representative, and its quantitative component was dominated by the public sector. It included 439 public entities, 39 private ones, and 19 social and industry organizations. So you can\u2019t automatically generalize the results to all Polish companies. However, they do show a mechanism that may also appear in business: tools are emerging faster than knowledge of their legal consequences.<\/p> Part of the problem probably stems from the fact that people still think of personal data mainly as a name, an ID number or an address. But an AI system might receive the contents of an email, a candidate’s CV, a customer complaint, notes from a meeting, a medical record, a conversation history or an excerpt from an internal report. Even if the user removes the name, the remaining information can still let someone identify the person.<\/p> Data also shows up in many places at once. It can be in the prompt you type into the model, an attached file, the conversation history, technical logs, the knowledge base connected to the system and the AI-generated response. Just because an organization uses an off-the-shelf tool doesn’t remove its responsibility for how information is entered and used.<\/p> The European Data Protection Board<\/mark><\/a> points out that you can’t assume upfront that an AI model is inherently anonymous and doesn’t process personal data. In practice, you need to check whether personal data can surface in its operation\u2014for example, whether the model was trained on such data or can reproduce it in its responses. You also assess whether appropriately crafted prompts could extract information about specific individuals from it. Even using an external model doesn’t automatically mean there’s no processing of personal data. Each such system should be assessed on a case-by-case basis.<\/p> That means choosing an off-the-shelf solution from a large vendor doesn\u2019t settle the matter. The organization still needs to know what data it\u2019s handing over, for what purpose, on what legal basis, where it\u2019s stored and who can access it.<\/p> Once the tool is up and running, some key decisions are already settled. The provider, hosting approach, licensing model and scope of integration have been chosen. Sometimes they\u2019ve also signed off on policies for retaining conversation history, using data to improve the service and transferring it outside the European Economic Area.<\/p> Bringing in a lawyer or a data protection officer later on doesn\u2019t always make it easy to reverse those decisions. Changing course may require renegotiating a contract, migrating data, reconfiguring the tool or dropping a feature around which the workflow has already been built.<\/p> The French data protection authority, CNIL \u2014 which has probably taken the most concrete approach to this issue \u2014 recommends analyzing risk and defining management rules before implementing generative AI. An organization should define its role and the provider\u2019s role, check the terms for data transfers and determine who\u2019s responsible for each operation. For cloud-based services, the scope of responsibility and access to data should be set out in the contract, not in the user\u2019s assumptions.<\/mark><\/a><\/p> In higher-risk applications, it may be necessary to conduct a data protection impact assessment, or DPIA. It\u2019s not just about creating a document in case of an audit. This analysis should show what risks arise, who they might affect and whether they can be mitigated before the system goes live.<\/p> Even a well-planned deployment still leaves one problem unsolved. An organization can control the system it purchased, but it doesn’t always know which tools its employees use.<\/p> The UODO report uses the term “shadow AI”\u2014the uncontrolled use of artificial intelligence outside official procedures. An employee might paste a document into a public chatbot to get a summary done faster. They might upload client data to generate a response, or use an external tool to analyze candidates, even though the company hasn’t formally approved it.<\/p> In such a situation, the organization might not even realize that data is now being processed in a new way. It doesn’t know the provider, hasn’t reviewed its terms, hasn’t established the legal basis and doesn’t know whether the information is being stored. The formal security policy covers one environment, while the actual work happens in a completely different one.<\/p> Banning the use of AI usually doesn’t solve this problem. It can only push it deeper into the gray area. We need approved tools, clear data policies and training that shows not only what not to do but also why a given use case creates risk.<\/p> One example of a reversed rollout is Snap\u2019s My AI chatbot. The service launched for paying Snapchat users in February 2023 and, less than two months later, for all users on the platform.<\/p>The AI is running, but nobody knows whether it’s processing data<\/h4>
Personal data isn’t just your social security number<\/h4>
The most important decisions are made at the point of purchase<\/h4>
The risk that remains in the shadows<\/h4>
When analysis arrives too late<\/h4>