Shadow AI in the workplace: what to do when company data has already landed in a chatbot

Monday morning, a mid-sized company somewhere in the German-speaking region. An intern wants to finish an internal analysis faster and quickly uploads a few documents to Claude – with their private, free account. Customer data included. The company policy is actually clear: only the approved Microsoft Copilot may be used. But the policy lives in a PDF that nobody has opened since onboarding.

The uncomfortable part: on free Claude accounts, using chats for model training has been enabled by default since September 2025. Anyone who didn't actively object when setting up the account has consented – and the data can sit in Anthropic's training pipeline for up to five years.

If you're thinking "that wouldn't have happened here": according to the IBM Cost of a Data Breach Report 2025, shadow AI was behind one in five of the data incidents studied. It happens. Constantly. The question is not whether, but how well your company is prepared for it.

What is shadow AI – and why is it more dangerous than shadow IT?

Shadow AI is the use of AI tools by employees without approval or control from IT. It ranges from a private ChatGPT account to browser extensions to AI features that have quietly crept into SaaS tools already in use.

The difference from classic shadow IT: an unapproved Dropbox merely stores your data. A consumer chatbot processes it – and can, depending on the provider and account setting, use it to train future models. Once something has flowed into a training run, no one gets it back. There is no recycle bin for training data.

The best-known case remains Samsung: in 2023, engineers handed confidential data to ChatGPT three times within just 20 days – proprietary source code, test sequences for chip manufacturing, internal meeting notes (t3n reported). Samsung reacted first with upload limits, then with a complete ban. What is remarkable is what came next: the ban did not solve the problem. Samsung has since moved to ChatGPT Enterprise – with a corporate contract, a guaranteed training exclusion, and mandatory security training before access is granted. The lesson in one sentence: anyone who wants to get rid of shadow AI has to offer an official path that is more convenient than the covert one.

The free-tier trap: why even a paid account doesn't automatically protect you

Here lies a detail that many companies misunderstand – and that makes the intern case so instructive.

Anthropic changed its consumer terms in August 2025: since then, chats and coding sessions are used for model training unless users actively object in the privacy settings. The catch: this applies not only to the Free plan, but equally to the paid Pro and Max subscriptions (TechCrunch, Steiger Legal). Anyone paying 20 dollars a month for Claude Pro still pays additionally with their data by default. With opt-in, data retention also extends from 30 days to five years.

Genuinely exempt from training are only the commercial offerings: Claude for Work, Enterprise, Education and API access fall under the Commercial Terms, which exclude training on customer data (Anthropic Privacy Center). At OpenAI the logic is similar: ChatGPT Free, Plus and Pro are consumer products with opt-out mechanics; only Team and Enterprise contracts offer the training exclusion as standard.

So the rule of thumb for your AI policy is not "paid = safe", but rather: consumer account = potential training data. Business contract with a data processing agreement = controlled processing. That is the dividing line your employees need to know.

Consumer account or business contract – the decisive dividing line

What risks is the company exposed to?

The figures from the IBM Cost of a Data Breach Report 2025 make the risk tangible: 20% of the companies studied suffered a data incident related to shadow AI. Incidents with a high shadow-AI share caused on average 670,000 US dollars in additional costs compared to ordinary incidents. In 65% of shadow-AI cases, personal customer data was affected – considerably more than the average across all breaches. And 97% of companies with AI-related incidents had no working access controls for AI.

Shadow AI risk in numbers – IBM Cost of a Data Breach 2025

Concretely, four categories of harm loom:

Data protection law (GDPR). If personal data is transmitted to an AI provider without a legal basis – without a data processing agreement under Art. 28 GDPR, possibly with a transfer to a third country – a notifiable breach is likely. Where there is a risk to data subjects, the 72-hour notification duty to the supervisory authority applies (Art. 33 GDPR); where the risk is high, there is also the duty to notify the data subjects (Art. 34).

Quick check – is the incident reportable under Art. 33/34 GDPR?

Trade secrets. Anyone who, as a holder of a secret, enters confidential information into a public chatbot may legally count as an infringer under trade secret protection – and the company may lose the protected status of the information, because "reasonable secrecy measures" are no longer in place (analysis by HÄRTING Rechtsanwälte on the Samsung case).

EU AI Act. Since 2 February 2025, Art. 4 of the AI Regulation has required companies deploying AI systems to ensure a sufficient level of AI literacy among their staff. Shadow AI is, by definition, undocumented AI use – if you don't know which systems are running in-house, you cannot meet this obligation.

Reputation and contractual penalties. NDAs with customers, industry requirements, certifications such as ISO 27001: a single upload can breach several contractual relationships at once.

When it has happened: these steps now

Back to the intern. The data has been uploaded – what needs to happen now, in this order?

Immediate measures – six steps in this order

1. Act immediately in the affected account. In the privacy settings, disable model training ("Help improve Claude" set to Off) and delete the affected conversations. Important for a realistic assessment: the opt-out only takes effect going forward. Data that has already flowed into an ongoing or completed training run will not be retrieved. The faster you act, the greater the chance that the content has not yet been used.

2. Contact the provider. Anthropic (like OpenAI) offers data protection contact channels for deletion requests. Invoke the right to erasure (Art. 17 GDPR) and document the account, time period and affected content as precisely as possible. It's not a guarantee – but a documented deletion request is worth gold later, towards both the supervisory authority and customers.

3. Assess the incident – within hours, not weeks. Which data categories were affected? Personal data, health data, trade secrets, customer data under NDA? That determines whether the 72-hour deadline under Art. 33 GDPR is running. Bring in your data protection officer or external advice before the deadline expires – not after.

4. Document. The sequence of events, the timings, the data affected, the measures taken. Even if there is no notification duty, Art. 33(5) GDPR requires internal documentation of every data breach.

5. Treat the person correctly. The intern is not the problem – they are the symptom. Punishing the messenger ensures that the next incident is kept quiet. The most valuable resource after such an incident is a reporting culture in which employees report mistakes immediately instead of covering them up.

6. Ask the systemic question. Why did they use Claude even though Copilot was approved? Most often the honest answer is: because the approved tool solved the task worse, more slowly, or not at all. That is the real insight you have to work with.

Can I check whether my data is already "in" ChatGPT and the like?

The honest answer: reliably, no – and be skeptical of anyone who sells you the opposite.

What lies behind this technically: large language models don't store documents, but statistical patterns. Individual inputs rarely lead to verbatim memorization – but it can't be ruled out, especially for unusual, frequently repeated or very specific content (credentials, unique identifiers, rare text fragments). You can deliberately ask a model about your content; a non-answer proves nothing, however, and a plausible answer may just as well be a hallucination. A negative "test" is worthless, a positive one hard to verify.

What you can do instead: exercise the right of access under Art. 15 GDPR towards the provider, request deletion under Art. 17, and – more pragmatically – treat compromised content as compromised: rotate credentials, inform affected customers depending on the risk situation, change internal identifiers. The energy belongs in damage control, not in the forensic search for the needle in the model.

How do you block unauthorized AI tools – and how are blocks circumvented?

The technical side first. Common measures, roughly sorted by effort:

• DNS/firewall blocking of the known AI domains (chat.openai.com, claude.ai, gemini.google.com …) – quick to implement, but crude.
• Secure web gateway / CASB with AI categories: detects and blocks AI services by category rather than by individual domain, including the constantly emerging new tools.
• Data loss prevention (DLP) at the endpoint and in the browser: it doesn't prevent the tool but the outflow – uploads and copy-paste of sensitive data patterns are detected and stopped. For data protection, the most effective layer.
• Browser management via group policies: whitelist extensions, centrally block unauthorized AI plug-ins.
• Access control at the identity level: approved AI tools only via SSO with the company account, so that private accounts don't work in the company context in the first place.

And now the uncomfortable truth: all of this has limits, and your employees know them. The usual workarounds are trivial – the private smartphone next to the keyboard (a photographed screen, retyped data), the private laptop in the home office, the mobile hotspot on the company device, private VPNs, or simply forwarding documents to a private email address. A Software AG study of 2,000 German knowledge workers put it plainly: 49% would keep using banned AI tools even if the company expressly prohibited them.

That's why: blocks are a hygiene measure, not a strategy. They shift the problem from the company network (where you have visibility) to private devices (where you're blind). Blocking alone doesn't make shadow AI smaller – only less visible.

The sustainable solution: an official path that is better than the covert one

What actually works is a combination of three building blocks:

First: a usable, approved alternative. People use shadow AI because they want to solve a real productivity problem. The offer has to keep up: a business contract with a training exclusion and DPA (Copilot, ChatGPT Team/Enterprise, Claude for Work) – or, where data sovereignty is a priority, a self-hosted solution with a local LLM where sensitive data never leaves the building in the first place. For many SMEs, exactly this combination is the sweet spot: cloud AI with a corporate contract for the non-critical, local AI for the crown jewels.

Second: an AI policy that fits on one page. Not the 40-page PDF nobody reads, but clear categories: which tools are approved (green), which are allowed only without sensitive data (yellow), which are off-limits (red)? Which data classes may go where? And: who do I turn to when I need a new tool – with a response time of days, not months? An approval process that takes a quarter is an invitation to shadow AI.

AI policy on a single page – the traffic-light system

Third pillar: awareness that doesn't annoy. The annual mandatory training with a multiple-choice test changes no behavior. What works: short, recurring nudges with real cases (the Samsung leak is more vivid than any slide), prompts right in the work context (such as a reminder when opening unapproved tools), onboarding moments – interns and new employees in particular are the high-risk group, because they haven't yet internalized the policies – and visible role-modeling by leadership. Since February 2025, by the way, this is no longer optional: Art. 4 of the EU AI Act makes AI literacy of the workforce mandatory. Documented, regular training is the evidence.

Is this a case for risk management?

Yes, clearly – and as a standalone risk position, not a footnote under "cyber". Here's how to build the case cleanly:

Risk identification: add "outflow of confidential data through unapproved AI tools" to the register as a named risk. Affected assets: personal data, trade secrets, customer data, source code.

Assessment: estimate the likelihood honestly – the research (20% of breaches, a majority of employees using their own tools) points to "high". Derive the potential loss from the GDPR fine framework, contractual penalties, incident costs (benchmark: the 670,000 USD in additional cost from the IBM report) and reputational damage.

Assign measures: the building blocks described above – approved alternative, policy, DLP, training – each with an owner, a deadline and an effectiveness review. Classically along the pattern avoid / reduce / transfer (keyword cyber insurance: check whether shadow-AI incidents are covered at all) / accept.

Monitoring: inventory AI use regularly (network discovery, SaaS audit), record incidents and near-misses, reassess annually. Anyone who has ISO 27001 or NIS2 in place hooks the risk in there – the structure already exists.

The intern incident itself belongs in the register as an incident: sequence of events, assessment, measures, lessons learned. Not as an assignment of blame, but as evidence that the company learns from incidents – which is exactly what auditors and supervisory authorities ask about.

Conclusion: shadow AI is a supply problem, not a prohibition problem

The intern with the free Claude account is neither an isolated case nor a character flaw. They are the predictable result of a gap between what employees need and what the company provides. Bans and blocks don't shrink this gap – they merely hide it.

The companies that have this topic under control do three things: they offer an official AI path that is genuinely usable. They draw a clear, understandable line between consumer accounts and contractually secured business solutions. And they treat AI literacy as an ongoing task rather than an annual box-ticking exercise.

Frequently asked questions

What is shadow AI? Shadow AI is the use of AI tools by employees without approval or control from the IT department – from a private ChatGPT account to an unvetted browser extension. Unlike classic shadow IT, these tools actively process data and, depending on the provider, may use it for model training.

Does Claude train on my company data? On the consumer plans (Free, Pro, Max), yes – by default, unless you object in the privacy settings. The commercial offerings (Claude for Work, Enterprise, API) are contractually excluded from training. The price of the account is not the criterion – the contract type is.

An employee entered sensitive data into a chatbot – what now? Immediately disable model training in the account and delete the chats, request deletion from the provider under Art. 17 GDPR, assess the incident under data protection law (check the 72-hour notification deadline under Art. 33 GDPR), document everything – and look for the cause in the system, not in the employee.

Is it enough to block ChatGPT and Claude on the company network? No. Blocks only work on the company network; according to a Software AG study, around half of German knowledge workers would keep using banned AI tools anyway – then via private devices, where the company has no visibility at all. What works is the combination of a usable approved alternative, a clear policy and DLP.

Are companies required to provide AI training? Yes. Art. 4 of the EU AI Act has required companies, since 2 February 2025, to ensure a sufficient level of AI literacy among staff operating AI systems. Documented, regular training serves as evidence.

Note: This article offers professional orientation but does not replace legal advice. For a binding assessment of your specific case, please consult qualified legal counsel.

---

Where does your company stand? If you're not sure which AI tools are already in use in your organization and which data is leaving the building in the process, that is precisely the first step: create visibility before the next upload happens. I support SMEs in the German-speaking region with a structured Sovereignty Check – from taking stock of actual AI use to architecting a sovereign, GDPR-compliant alternative that your employees will use voluntarily. → Request a no-obligation initial call

Sources

• IBM Newsroom: Cost of a Data Breach Report 2025 – key findings on AI and shadow AI
• IBM Think: What is shadow AI?
• TechCrunch: Anthropic users face a new choice – opt out or share your chats for AI training
• Anthropic Privacy Center: Is my data used for model training?
• Steiger Legal: Opt-out instead of opt-in – Anthropic uses user data for AI training by default
• t3n: Data leak at Samsung – engineers send confidential data to ChatGPT
• HÄRTING Rechtsanwälte: Samsung's ChatGPT leak – AI risks in everyday work (trade secret protection)
• IT Finanzmagazin: Schatten-KI – das unterschätzte Risiko: Robuste KI-Governance notwendig (on the Software AG study "Chasing Shadows – Getting Ahead of Shadow AI", German)