Enterprise buyers already know how to review software security: authentication, encryption, access control, compliance, penetration testing. AI systems pass all of that and still introduce risks a standard questionnaire never asks about. The teams closing enterprise AI deals are the ones who can answer the AI-specific questions — the ones about the new attack surface that a language model brings with it. Here is what serious buyers now check, on top of the usual application security review.
Why AI needs its own security review
A traditional app does what its code says. An AI system takes untrusted natural-language input, reasons over data, and produces open-ended output — which means new ways to attack it and new ways for it to fail. A perfectly "secure" application can still leak data or take a harmful action because the model was manipulated. That gap is exactly what AI-aware buyers probe.
What buyers actually check
- Prompt injection defence. Can a malicious instruction hidden in a document, email or web page hijack the model's behaviour? Buyers ask how you separate trusted instructions from untrusted content and how tool-use is constrained. This is the AI equivalent of SQL injection — see prompt injection.
- Data access and leakage. What can the model actually see? With retrieval-augmented generation, retrieval must respect the same permissions as the user asking — otherwise the AI becomes a way to read data someone was never allowed to see. Buyers check that retrieval is permission-aware and that sensitive fields are redacted.
- Where the data goes. Does our data leave our boundary? Is it sent to a third-party model provider, retained, or used to train their models? Data residency and a no-training guarantee are now standard asks.
- Output safety and guardrails. What stops a confident wrong answer or a harmful action? Buyers look for output validation, guardrails, and controls against hallucination — especially where the AI can trigger real actions rather than just talk.
- Access control and audit. Who can query the system, and is every interaction logged? Enterprises need an audit trail of prompts, retrieved sources and responses for incident review and compliance.
- Human oversight and fail-safes. For consequential decisions, is there a human in the loop and a safe fallback when the model is uncertain? "The AI decided" is not an acceptable answer in a regulated process.
- Model governance. Which models are used, how are changes reviewed, and how do you evaluate for regressions and bias over time? Buyers increasingly ask this to satisfy emerging AI regulation.
The pattern behind the questions
Every question above comes back to one principle: treat the model as an untrusted, powerful component — never give it more data or more ability to act than the situation requires, and verify what it does. Security designed in from the architecture stage answers these questions naturally. Security bolted on after a pilot usually means a rebuild, which is one more reason pilots stall before production — a theme we cover in why enterprise AI pilots stall.
How to be ready
If you are buying, put these questions in your evaluation early — a vendor who cannot answer them is showing you a demo, not a system. If you are building, document your answers before procurement asks: how you defend against prompt injection, how retrieval respects permissions, where data goes, what guardrails exist, and how you log and oversee the system. That document shortens security review from weeks to days.
Getting AI security right is not a checklist you add at the end — it is part of how the system is engineered from day one. That is the approach behind our AI development work and our wider view on enterprise AI strategy.