Testing Beyond Checks
Passing Checks ≠ Understanding Risk
My ConfQ 2026 Talk
Last week, I had the privilege of speaking at ConfQ 2026 on a topic that has shaped much of my thinking over the past few years:
Testing Beyond Checks: Why Passing Checks Is Not the Same as Understanding Risk.
The session wasn’t an argument against automation.
It wasn’t an argument against CI/CD pipelines.
And it certainly wasn’t an argument against checking.
Checking is essential.
Automation is essential.
Fast feedback is essential.
The question I wanted to explore was different:
What happens when checking quietly becomes our definition of testing?
That question has influenced many of the articles I’ve written here on LifeOfQA. At ConfQ, I had the opportunity to bring those ideas together into one story.
🎥 Watch the presentation below:
The central idea
If I had to summarize the entire presentation in one sentence, it would be this:
Evidence of activity is not evidence of understanding.
Modern engineering teams generate an incredible amount of evidence.
Coverage reports.
Passing automated checks.
Green pipelines.
Defect dashboards.
Release approvals.
These are all useful signals.
But none of them, on their own, tell us whether we truly understand the risks within our system.
Passing checks tells us that certain expectations were verified.
Testing asks a different question:
Were our expectations sufficient in the first place?
That distinction may seem small, but it changes how we think about quality.
What teams really want
No engineering team wakes up hoping to execute more test cases.
Nobody celebrates increasing code coverage by another one percent.
Those things are tools, not goals.
What teams really want is confidence.
Confidence to release.
Confidence to make changes.
Confidence to move faster.
Confidence to make better decisions.
The problem is that confidence can come from two very different places.
One is built on genuine understanding.
The other is built on signals that simply make us feel safe.
Those are not the same.
When confidence becomes dangerous
During the talk, I discussed two well-known incidents: Knight Capital and CrowdStrike.
Different companies.
Different technologies.
Different industries.
Different years.
Yet they revealed a remarkably similar pattern.
The checks worked.
The assumptions failed.
That gap between what we verified and what we truly understood is where many of our biggest software risks hide.
These examples aren’t meant to criticize those organizations.
They’re reminders that even highly capable teams can miss important risks when confidence becomes disconnected from understanding.
Why smart teams still fall into this trap
I don’t believe these failures happen because engineers stop caring.
More often, they’re influenced by normal human psychology.
In the talk, I discussed three ideas that explain why intelligent teams can gradually become overconfident:
Automation Bias – trusting automated results more than our own judgment.
Goodhart’s Law – when a measure becomes a target, it stops being a good measure.
McNamara Fallacy – focusing only on what is easy to measure while ignoring what matters most.
Automation is valuable.
Metrics are valuable.
Dashboards are valuable.
But none of them should replace critical thinking.
So what does testing actually do?
For me, testing isn’t primarily about execution.
Testing is investigation.
Testing is learning.
Testing is reducing uncertainty.
Checking confirms what we already expect.
Testing explores whether those expectations are complete.
Whenever I begin exploring a new product or feature, I keep returning to three simple questions:
What assumptions are we making?
What could seriously hurt our users?
What behavior would genuinely surprise us?
These aren’t test cases.
They’re thinking tools.
Good testing often begins with a better question, not a bigger checklist.
Checking and testing need each other
One misunderstanding I wanted to avoid during the session was the idea that this is somehow an argument against checking.
It isn’t.
Modern software development depends on checking.
Without automated checks, continuous delivery at today’s scale wouldn’t be possible.
But checking and testing perform different jobs.
Checking provides repeatable evidence.
Testing creates understanding.
Quality needs both.
Testing is a social activity
One of the ideas closest to my heart is that testing is not just a technical activity.
It’s a social one.
Quality doesn’t come from a QA team alone.
It emerges from the decisions developers, testers, product managers, architects, designers, and leaders make together.
Testing contributes information.
It creates understanding.
It helps teams make better decisions.
That’s why I believe risk doesn’t live inside test cases.
Risk lives in conversations.
One question I encourage teams to replace in release meetings is this:
“Did all the tests pass?”
It’s a useful question.
But it’s incomplete.
Instead, ask:
“What risks remain?”
That single question changes the conversation from reporting activity to discussing uncertainty.
Final thoughts
If there’s one idea I’d like readers to remember, it’s this:
Confidence is not earned by passing checks. Confidence is earned by understanding risk.
The software industry doesn’t need fewer checks.
We need thoughtful checking.
Thoughtful automation.
Thoughtful engineering.
But above all, we need curious people who continue asking questions that dashboards cannot answer.
Because checks provide evidence.
Testing creates understanding.
And quality needs both.
Further Reading
If you found this useful, you might also enjoy these articles:
If you found this helpful, stay connected with Life of QA for more real-world testing experiences, tips, and lessons from the journey!