AI Security Checklist 2025

LLM Security Threats

Prompt injection involves manipulating LLM behavior through crafted inputs that override system instructions. This includes direct injection (user input) and indirect injection (external data like emails, documents, web pages).

Real-World Attacks

The document illustrates documented attack cases including SpAIware attacks, Slack data exfiltration, and EchoLeak vulnerabilities.

Advanced Attack Techniques

Modern prompt injection has moved well beyond basic instruction overrides. Current research documents evasion techniques that bypass production safeguards, including adversarial poetry methods achieving “62% jailbreak rates across frontier models.”

Other Documented Techniques

Additional emerging attack vectors are documented alongside critical insights about the evolving threat landscape.

Insecure Output Handling

LLM outputs processed downstream without validation enable injection attacks (XSS, SQL, command injection, SSRF) through model outputs. Real examples include LangChain and Vanni vulnerabilities.

Security Checklist

Recommendations include output sanitation and sandbox execution for generated code.

Mitigation Strategies

1. Defense in Depth: Layer input validation + output filtering + privilege limits
2. Regular Testing: Quarterly red team exercises with adversarial prompts
3. Monitoring: Alert on unusual patterns and system keywords

Monitoring & Response

Comprehensive approaches to input monitoring, output monitoring, and incident response procedures are essential.

Quick Reference

Critical Actions

WEEK 1 - Immediate

1. Scan repos for exposed API keys
2. Rate limit all AI APIs
3. Add PII detection on outputs
4. Enable input/output logging
5. Restrict model permissions

MONTH 1 - Foundation

1. AI security risk assessment
2. Prompt injection detection
3. Output validation
4. Monitoring and alerting
5. Incident response plan

QUARTER 1 - Maturity

1. Comprehensive monitoring
2. LLM-specific pen testing
3. Adversarial robustness testing
4. Model governance
5. Staff training

Conclusion

The document emphasizes that “we’re currently in the infancy of AI, and AI security.” Unknown risks will require “a combination of technology and humans” to identify effectively, as AI tools trained on known data may miss emerging vulnerabilities.