AI Security: Understanding the OWASP Top 10 for LLM Applications

The proliferation of Large Language Models (LLMs) in enterprise applications - from customer-facing chatbots and code generation tools to internal knowledge retrieval and clinical decision support systems - has introduced a fundamentally new attack surface that traditional application security frameworks were never designed to address. The OWASP Top 10 for LLM Applications (v2.0, 2025) provides the most comprehensive threat taxonomy for these systems and should be the baseline security framework for any organization deploying LLM-powered features in production.

LLM01: Prompt Injection

Direct vs. Indirect Injection

Prompt injection remains the most critical and most exploited vulnerability in LLM applications. It occurs in two forms: direct prompt injection, where a user crafts input to override or manipulate system instructions, and indirect prompt injection, where malicious instructions are embedded in external data sources (documents, web pages, API responses) that the LLM processes.

Multi-Turn and RAG Poisoning Attacks

Direct injection attacks range from simple instruction override ('Ignore all previous instructions and...') to sophisticated multi-turn manipulation that gradually shifts the model's behavior over a conversation. Indirect injection is particularly dangerous in Retrieval-Augmented Generation (RAG) systems, where an attacker can poison documents in the knowledge base with hidden instructions that execute when retrieved.

Defense-in-Depth Layers

Defense-in-depth for prompt injection requires multiple layers: input sanitization (strip or encode control characters, detect known injection patterns), output validation (verify outputs against expected schemas and content policies), privilege separation (never allow the LLM to directly execute code, database queries, or API calls without a validation layer), instruction hierarchy (use system-level guardrails that the model is fine-tuned to prioritize over user input), and canary token detection (embed hidden markers in system prompts to detect extraction attempts).

LLM02: Insecure Output Handling

LLM Output as Injection Vector

When LLM outputs are consumed by downstream systems without sanitization, the model becomes a vector for traditional injection attacks. If model output is rendered as HTML, you're vulnerable to XSS. If it's interpolated into SQL queries, you've created a SQL injection pathway. If it's used to construct system commands, you're exposed to OS command injection.

Risks in Agentic Architectures

This vulnerability is especially insidious because developers often trust model output as 'computed' rather than 'user-supplied,' bypassing standard input validation. In agentic architectures where models call tools or APIs, insecure output handling can lead to Server-Side Request Forgery (SSRF), arbitrary file access, or privilege escalation through crafted tool arguments.

Treating Output as Untrusted Input

Mitigation requires treating all LLM output as untrusted input: apply context-specific output encoding (HTML encoding for web rendering, parameterized queries for database operations), implement allowlists for tool/function arguments, sandbox code execution environments, and use Content Security Policy (CSP) headers to limit the impact of any injected scripts.

LLM03: Training Data Poisoning

Backdoors via Poisoned Training Data

Training data poisoning is a supply-chain attack targeting the model's learning process. By injecting malicious, biased, or manipulated data into training or fine-tuning datasets, attackers can influence model behavior in ways that are extremely difficult to detect through standard testing. For fine-tuned models, even a small percentage of poisoned examples (as low as 0.1% of the training set in some research) can create exploitable backdoors.

RAG Corpus Poisoning

For RAG-based systems, corpus poisoning is the equivalent threat. Attackers inject carefully crafted documents into the retrieval corpus - these documents contain plausible-looking content mixed with malicious instructions or disinformation. When retrieved and fed to the model as context, these poisoned documents can manipulate outputs, bypass safety guardrails, or exfiltrate sensitive information from the prompt context.

Provenance and Adversarial Testing

Defenses include: rigorous data provenance tracking and validation for all training data, statistical analysis of training data distributions to detect anomalies, regular evaluation of model outputs against known-good baselines, integrity verification of RAG corpus documents (hash-based verification, access controls, anomaly detection on corpus changes), and adversarial testing specifically targeting data poisoning scenarios.

LLM04: Model Denial of Service

Asymmetric Compute Cost Attacks

LLM inference is computationally expensive - a single complex prompt can consume orders of magnitude more resources than a typical API request. Attackers exploit this asymmetry through crafted prompts that maximize token generation (recursive or self-referential prompts), requests that trigger maximum context window utilization, batch requests designed to exhaust GPU compute quotas, and prompt sequences that cause the model to enter degraded performance states.

Mitigation: Limits and Circuit Breakers

Mitigation requires: strict input length limits (enforce maximum token counts before the request reaches the model), per-user and per-IP rate limiting with exponential backoff, request timeout enforcement at the inference layer, cost monitoring and automatic circuit breakers when spend thresholds are exceeded, and queue-based architectures that degrade gracefully under load rather than failing catastrophically.

LLM05: Supply Chain Vulnerabilities

Complex LLM Supply Chain

The LLM supply chain is uniquely complex: it encompasses base model providers, fine-tuning data pipelines, embedding model dependencies, vector database infrastructure, prompt templates, plugin/tool ecosystems, and inference infrastructure. Each link represents a potential compromise point.

Specific Supply Chain Risks

Specific risks include: compromised model weights from untrusted sources (model repositories can host backdoored models), vulnerable dependencies in inference frameworks (e.g., deserialization vulnerabilities in model loading), insecure plugin architectures that grant excessive permissions to third-party extensions, and prompt template injection through shared prompt repositories.

Hardening Strategies

Supply chain hardening requires: verifying model provenance and integrity (cryptographic signatures, checksums), maintaining a software bill of materials (SBOM) for your entire AI stack, vetting and sandboxing all plugins and extensions, pinning dependency versions and monitoring for CVEs, and maintaining the ability to rapidly switch model providers if a supply chain compromise is detected.

LLM06-LLM10: Additional Critical Risks

LLM06: Sensitive Information Disclosure

Sensitive Information Disclosure (LLM06): Models can memorize and leak training data, including PII, credentials, and proprietary information. Implement output filtering for known sensitive data patterns (SSNs, API keys, email addresses), use differential privacy techniques during training, and deploy PII detection classifiers on model outputs.

LLM07: Insecure Plugin Design

Insecure Plugin Design (LLM07): Plugins that extend LLM capabilities often lack proper input validation, authentication, and authorization. Apply the principle of least privilege to all plugin permissions, validate all parameters passed from the LLM to plugins, and implement human-in-the-loop approval for high-impact actions (financial transactions, data deletion, external communications).

LLM08: Excessive Agency

Excessive Agency (LLM08): LLM-powered agents with broad tool access can perform unintended actions. Limit the scope and impact of each available tool, implement confirmation workflows for destructive operations, log all tool invocations for audit, and use allowlists rather than blocklists for permitted actions.

LLM09: Overreliance

Overreliance (LLM09): Organizations that deploy LLM outputs without human oversight in critical decision-making processes risk propagating hallucinations, biases, or errors at scale. Implement confidence scoring, citation verification for factual claims, and mandatory human review for high-stakes outputs.

LLM10: Model Theft

Model Theft (LLM10): Extraction attacks can reconstruct model capabilities through systematic querying. Implement query rate limiting, monitor for extraction patterns (high-volume systematic queries), watermark model outputs, and restrict API access to authenticated users with usage quotas.

Building an LLM Security Program

Structured Security Program

Organizations deploying LLMs should implement a structured security program: conduct threat modeling specific to your LLM architecture (RAG, fine-tuned, agentic), perform regular red-team exercises using the OWASP LLM Top 10 as a testing framework, integrate LLM-specific security testing into your CI/CD pipeline, monitor model behavior in production for drift and anomalies, maintain an incident response plan that addresses LLM-specific breach scenarios, and establish an AI governance committee that includes security, legal, and ethics stakeholders.

Continuous Assessment

The attack surface for LLM applications is evolving rapidly. What constitutes a best practice today may be insufficient tomorrow. Continuous security assessment, threat intelligence monitoring, and proactive defense updates are essential for maintaining a robust security posture around your AI deployments.