Securing the AI Supply Chain: Risks in ML Model Pipelines

The Expanding AI Attack Surface

The adoption of AI across enterprise operations has created a new category of supply chain risk. The AI supply chain encompasses pre-trained models, training datasets, data preprocessing pipelines, model serving infrastructure, and the MLOps tooling managing the full lifecycle. Each component presents distinct attack surfaces.

Attack Vectors in the AI Supply Chain

Compromised Pre-Trained Models

The open-source AI ecosystem has normalised downloading pre-trained models from repositories — Hugging Face, GitHub, cloud provider model hubs — and fine-tuning them for specific use cases. A compromised model that appears legitimate but contains embedded backdoors represents a threat that standard software security scanning doesn't detect.

Model backdoor attacks embed hidden triggers causing the model to behave maliciously when specific input patterns are encountered — a security classifier backdoored to classify specific malware as benign when a trigger pattern is present is a realistic threat.

Poisoned Training Data

Models learn from data, and compromised data produces compromised models. Training data poisoning introduces carefully crafted samples causing trained models to develop specific, exploitable flaws.

For security models trained on shared threat intelligence feeds or publicly available malware repositories, supply chain compromise of these data sources is a viable poisoning vector.

Compromised MLOps Tooling

The tooling managing the ML lifecycle — experiment tracking, model versioning, pipeline orchestration — is subject to the same supply chain risks as any software dependency. Compromised MLOps tools can modify models during training or inference, exfiltrate training data, or introduce subtle performance degradation.

Inference Infrastructure Attacks

Models deployed as APIs present attack surfaces similar to traditional web services plus model-specific risks including model extraction attacks (systematically querying the API to reconstruct a local copy) and prompt injection through crafted inputs.

Security Controls for AI Supply Chain

Model provenance and integrity verification. Cryptographic signing and verification of pre-trained models before use, evaluating models only from verified sources with established security practices.

Training data governance. Maintain control over training data sources with integrity verification of dataset downloads and anomaly detection on data distribution to identify potential poisoning.

Dependency scanning for ML packages. Extend software composition analysis to cover the ML-specific package ecosystem — PyTorch, TensorFlow, scikit-learn, and their dependencies.

Sandboxed model evaluation. Evaluate pre-trained models in isolated environments before production deployment. Behavioural testing that probes for backdoors and anomalous resource usage can identify compromised models before they reach production.