Generative AI and Data Leakage: What CISOs Need to Know

The Generative AI Data Exposure Problem

Employees are sharing sensitive information with AI tools at unprecedented scale, often without awareness of the implications. Security teams have documented: sensitive customer data submitted to public AI tools for analysis, internal financial information used to draft investor communications, personnel data used in AI-assisted HR processes, source code submitted to AI coding assistants.

Each interaction potentially exposes sensitive data to AI service providers' training processes, staff access, and security incidents affecting those providers.

The Risk Landscape

Training Data Exposure

Many public AI services train on user-submitted data unless users explicitly opt out — and even with opt-out, data handling practices vary by provider. Sensitive data submitted to AI services may persist in training datasets, potentially influencing model outputs for other users.

Third-Party Data Processing

Submitting data to AI services invokes their data processing terms, potentially in conflict with your organisation's data classification policies, contractual obligations, and regulatory requirements including GDPR.

Prompt Injection in Business Processes

AI systems integrated into business workflows are vulnerable to prompt injection — malicious instructions embedded in data that the AI processes, causing it to perform unintended actions. An AI-powered email assistant could be manipulated by a malicious sender to exfiltrate information or modify its behaviour through embedded instructions.

CISO Strategy: Enable and Control

The organisations that handle generative AI risk best are those that enable productive use through controlled channels rather than attempting prohibition. Prohibition without alternatives drives shadow AI use that is entirely outside organisational visibility or control.

Establish an approved AI tool catalogue. Identify AI services evaluated for data handling practices, security posture, contractual terms protecting organisational data, and regulatory compliance.

Implement data classification training for AI use. Public AI tools should only receive public or internal data. Confidential or restricted data should only be submitted to enterprise AI services with appropriate data protection terms.

Deploy AI activity monitoring. Data loss prevention tools adapted for AI interaction monitoring, browser-based controls restricting data submission to unapproved AI services, and API gateways logging AI service interactions provide governance visibility.

Engage with enterprise AI procurement. Microsoft Copilot, AWS Bedrock, Azure OpenAI Service provide data protection terms that public AI services don't. Understanding the enterprise AI landscape and where it meets your risk requirements enables AI use within acceptable parameters.