Evaluating Security Posture

Last updated: 2026-05-18

Evaluating an AI Tool's Security Posture

Security due diligence before adopting an AI tool isn't optional. A vendor with weak controls is a liability, regardless of how useful the product is. This guide covers what to check, what documentation to ask for, and what red flags should make you walk away.

SOC 2 Certification

SOC 2 is an audited framework covering security, availability, processing integrity, confidentiality, and privacy. Type I covers whether controls are designed correctly. Type II confirms they've been operating effectively over time.

It indicates the vendor has invested in documented, tested security practices. Enterprise buyers typically require it.

When evaluating: ask whether they have SOC 2, which type, how recently it was audited, and whether they can share a summary or report under NDA.

Encryption

Data at rest should be encrypted, typically AES-256. Data in transit should use TLS 1.2 or 1.3. Unencrypted HTTP for anything sensitive is a non-starter.

Beyond the standard, ask about key management: who holds the keys, how often they're rotated, and whether they offer bring-your-own-key (BYOK) for organizations with strict requirements.

Data Residency

Data residency means where your data is actually stored. This matters for GDPR compliance, national data sovereignty requirements, and cross-border transfer rules.

Ask where data is stored by default, whether you can choose a region, and whether they support your specific residency requirements.

Retention Policies

Retention policy determines how long a vendor keeps your data. Longer retention means longer exposure. Short, configurable retention with auto-deletion is better.

Ask what the default retention period is, whether you can set a shorter one, and whether you can export data before deletion occurs.

Access Controls

Proper access controls limit who inside the vendor organization can reach your data.

Look for MFA, SSO support, role-based permissions, audit logs for data access, and a principle of least privilege throughout their internal systems.

Penetration Testing

Regular penetration tests by independent third parties identify vulnerabilities before attackers do. Self-assessments are less credible.

Ask how often they test, who does it, and whether they can share a summary of findings and how they were addressed.

Incident Response

How a vendor responds to a breach matters as much as whether one occurs.

They should have a documented incident response plan, a defined process for notifying customers, a stated SLA for breach notification, and cyber insurance. Ask to see the outline.

Security Checklist

Red Flags

Walk away or escalate scrutiny if you see any of these: