What the insurance pullback says about AI risk
CIO reported on April 20, 2026 that some insurance carriers are backing away from covering AI outputs in cyber and errors and omissions policies. Some are adding exclusions, some are raising prices, and some are asking more detailed questions about how companies use AI.
The article does not say every carrier is refusing AI coverage. The better reading is more practical. Insurers are trying to understand which AI uses are governed, which are experimental, and which are too hard to price.
That matters for software leaders because AI risk is moving beyond engineering. It is showing up in underwriting questions, governance reviews, legal reviews, customer risk reviews, and board discussions.
The question is changing: Do you use AI? is giving way to Can you show how your AI is governed, tested, monitored, and controlled?
Why this becomes a testing problem
AI risk often looks like a governance issue. Underneath it, there is usually a testing problem.
Can the team show what the AI system was expected to do? Can it show how the system was tested before release? Can it show known failure modes? Can it show what happens when outputs are wrong? Can it show that the system is monitored after launch?
Those are testing questions. They are also assurance questions. This is where AI testing becomes part of business control, not just QA execution.
Short answer: AI testers do not prove that AI is perfect. They help teams prove that AI risk was identified, tested, reduced, monitored, and communicated.
What AI testers add when AI outputs create business risk
When AI output can affect customers, operations, legal duties, or security, testing has to go beyond it looked right in a demo. AI testers add structure.
Define expected behavior
AI testers help teams define what acceptable behavior looks like. That includes normal use, edge cases, misuse, and cases where the AI should refuse, ask for help, or escalate to a human.
Build repeatable eval sets
Instead of testing with random prompts, AI testers build reusable evaluation sets. These help teams compare behavior across model updates, prompt changes, retrieval changes, and product releases.
Test known failure modes
AI testers check hallucination, bias, prompt injection, insecure output handling, sensitive data exposure, unsafe tool use, overreliance, and weak human review. The OWASP Top 10 for Large Language Model Applications is a useful reference for several of these LLM risks.
Document risk decisions
The goal is not to claim that AI is risk-free. The goal is to show what was tested, what failed, what was fixed, what remains risky, and who accepted that risk.
Monitor after launch
AI behavior can change when data, prompts, retrieval sources, models, users, or workflows change. AI testers help teams turn production examples into better monitoring and regression coverage.
Explain risk in business language
Executives, insurers, customers, and auditors do not need a wall of model jargon. They need clear evidence about impact, controls, limits, and response plans.
Governed AI is easier to defend than experimental AI
The CIO article draws a useful line between governed AI and more experimental AI deployments. That distinction matters.
A chatbot with clear use limits, test coverage, monitoring, logging, and rollback plans is very different from an autonomous workflow that can take action with little oversight.
AI testers help companies move from we tried it and it seemed fine to we know what this system is allowed to do, how it was tested, what risks remain, and how we will catch problems.
That is the kind of evidence leaders need when AI shows up in insurance, security, compliance, and customer conversations.
Governed AI usually has
- A clear use case
- Known users
- Defined limits
- Test evidence
- Human review for risky outputs
- Monitoring, logs, and a rollback plan
Experimental AI often has
- A vague use case
- Unclear output limits
- Little test evidence
- Weak review
- Poor logging
- No clear owner or rollback plan
Vibe-coded systems make the testing gap worse
Vibe coding and AI-assisted coding can move fast. That is useful, but it can also create hidden risk.
Code may look clean while the design is weak. Tests may look complete while they miss important cases. Documentation may sound confident while it hides uncertainty.
That is why AI-generated code and AI-assisted workflows still need independent review. A tester should ask:
- What did the AI generate?
- What did a human verify?
- What was tested directly?
- What was assumed?
- What could fail in production?
- What would we show if a customer, auditor, insurer, or executive asked?
Fast development does not remove the need for testing. It raises the value of testers who can slow down at the right moment and find the risk before it becomes expensive.
Where AI Assurance Pro fits
AI Assurance Pro does not make an AI system insurable by itself. No certification can do that.
What it does is give testers and teams a structured knowledge path for the work that insurers, executives, and customers are starting to ask about.
ASTQB AI Assurance Pro™ is based on three ISTQB certifications: Foundation Level, ISTQB AI Testing, and ISTQB Testing with Generative AI. The path combines testing fundamentals, testing AI-based systems, and using generative AI in software testing work.
That matters because AI assurance is not only about tools. It is about people who know how to test, question, measure, document, and communicate AI risk. NIST describes trustworthy AI using qualities such as valid and reliable, safe, secure and resilient, explainable, privacy-enhanced, and fair with harmful bias managed in its AI Risk Management Framework resources.
ISTQB Foundation Level
Builds the shared testing base: risk, test design, defects, coverage, reporting, and test process.
ISTQB AI Testing
Covers testing AI-based systems, including bias, non-determinism, data quality, model metrics, explainability, drift, and AI-specific test strategy.
ISTQB Testing with Generative AI
Covers generative AI in software testing, including prompting, risks, use cases, LLM-powered test infrastructure, and test organization strategy.
ASTQB AI Assurance Pro™
Brings the three-part path together into a designation that helps managers and teams identify testers with structured AI testing knowledge.
If you want the credential path, start with the three required certifications, then read how to get AI Assurance Pro. For the team view, go to AI Assurance Pro for managers.
Questions every AI project should answer before release
A team does not need a perfect answer to every question before it starts using AI. But before AI reaches customers, production workflows, or high-risk business processes, these questions should not be vague.
- What business process does this AI system affect?
- What could go wrong if the output is wrong?
- Who reviews high-risk outputs?
- What data can the system access?
- Can the system take action, or only suggest action?
- How do we test for hallucination, bias, misuse, and security issues?
- How do we test prompt injection and unsafe output handling?
- What logs or records show how the system behaved?
- What happens if the model, prompt, or retrieval source changes?
- How do we monitor production behavior?
- What is the rollback plan?
- Who owns the risk decision?
These are not only policy questions. They are testing questions. A trained AI tester helps turn them into evidence.
What managers should look for in AI testers
The best AI testers do not just know AI terms. They can explain how they would create evidence.
A manager should look for people who can define risk, design evaluation sets, challenge AI output, review AI-generated tests, explain failure modes, and communicate findings without hiding behind model jargon.
Ask for a real test strategy
Give the candidate a chatbot, summarizer, search assistant, recommendation feature, or AI coding workflow. Ask how they would test it before release.
Ask how they handle non-determinism
Strong answers define acceptable variation, repeated runs, thresholds, risk levels, and escalation rules.
Ask about hallucination and groundedness
Look for source checks, evidence checks, confidence limits, retrieval validation, and human review where the risk is high.
Ask them to review AI-generated tests
This shows whether they can use AI as a tool without trusting it blindly. For a deeper hiring guide, see how to evaluate AI testing skills when hiring.