Learning From Failure

Failure Can Become Information

Failure is costly when nothing is learned from it. A prediction can fail, an intervention can produce weaker results than expected, a policy can create unintended consequences, or a well-designed plan can encounter conditions that were not understood in advance. Each outcome contains information.

WIN treats failure analysis as part of disciplined learning. The objective is neither to excuse poor performance nor to search automatically for someone to blame. It is to determine what happened, why expectations were not met, what the result reveals about the underlying explanation or implementation, and what should change before another attempt is made.

A system that records only its successes can become increasingly confident while becoming less accurate. A system that examines failure carefully has a better chance of correcting itself.

Different Forms of Failure Reveal Different Things

Not every disappointing result has the same meaning.

An explanation may be wrong. The explanation may be substantially correct but incomplete. Implementation may differ from the design. Conditions may change. Resources may be insufficient. The measurement may be poor. The intended benefit may occur while a serious unintended cost appears elsewhere.

Separating these possibilities matters. If an idea failed because its central assumption was false, improving execution will not repair the explanation. If the explanation was sound but implementation broke down, discarding the underlying idea may waste useful knowledge.

Failure analysis therefore begins by determining what actually failed.

Compare the Result With the Expectation

Learning requires knowing what was expected before the outcome was known.

What was supposed to change? By how much? Over what period? Under what conditions? What secondary effects were anticipated? What observations would have indicated that the intervention was not working?

When expectations are explicit, the difference between prediction and result becomes evidence. Without a prior expectation, almost any outcome can be explained afterward, making weak ideas difficult to challenge.

WIN therefore connects failure analysis to its broader measurement principle: meaningful evaluation begins with sufficiently clear expectations.

Distinguish Concept Failure From Implementation Failure

A disappointing result does not automatically reveal whether the underlying idea or its implementation was responsible.

A concept can fail because an important causal assumption was wrong. Implementation can fail because procedures were unclear, training was inadequate, resources were unavailable, technology malfunctioned, participants did not receive the intended intervention, or conditions differed materially from the design.

WIN examines both possibilities.

Implementation problems are not used automatically to rescue a preferred theory. If repeated competent implementations continue to produce poor results, confidence in the underlying explanation should decline.

Conversely, an implementation failure does not automatically establish that the underlying concept was invalid.

Avoid the Blame Trap

Accountability matters, but blame by itself does not explain a failure.

If analysis stops with “someone made a mistake,” important questions remain unanswered. What conditions made the mistake more likely? Was information available? Were responsibilities clear? Was training adequate? Were procedures understandable? Did incentives encourage the wrong behavior? Were safeguards missing? Was workload unreasonable? Could the system have detected the error sooner?

Useful analysis examines individual responsibility and system design at the same time.

The objective is not to eliminate personal accountability. It is to prevent an individual explanation from concealing a correctable system weakness.

Distinguish Good-Faith Error From Misconduct

A research and learning system should not treat every error as wrongdoing.

People can make reasonable decisions from incomplete information and still be wrong. If every mistake is punished as misconduct, people gain an incentive to hide uncertainty, conceal failure, manipulate measurements, or avoid reporting problems.

Deliberate deception, evidence manipulation, concealment, reckless disregard, retaliation, and conflicts of interest are different problems. They require separate examination because they affect the reliability of the information itself.

Learning depends on preserving the distinction between good-faith error and misconduct.

Look for Unintended Consequences

An intervention can improve the measure being watched while making another part of the system worse.

Costs can shift to another group, another institution, or a later period. Incentives can encourage people to optimize the measurement rather than the real objective. A short-term success can weaken long-term resilience. A benefit for one population can create an unacceptable burden for another.

WIN therefore evaluates both intended and material unintended outcomes.

The broader question is not merely, “Did the target number improve?” It is “What happened to the relevant system as a whole?”

Examine Near Misses

Learning does not require waiting for a complete failure.

A near miss—an event in which a serious problem almost occurred but was prevented by chance, late intervention, redundancy, or another protective factor—can expose a weakness before substantial harm occurs.

WIN treats consequential near misses as information about vulnerabilities, assumptions, safeguards, procedures, or system design.

Correcting a weakness discovered through a near miss is preferable to waiting for the same weakness to produce a larger failure.

Ask Whether the Failure Was Predictable

After a failure occurs, warning signs can appear obvious in hindsight.

Responsible analysis asks what information was actually available before the outcome occurred. Were warning signs present and reasonably recognizable? Were concerns raised but ignored? Did existing measurements fail to detect the problem? Was the risk known but judged acceptable? Or did the failure arise from conditions that could not reasonably have been anticipated?

This distinction helps separate genuine learning from hindsight bias.

Search for the Failure Chain

Important failures often result from several conditions interacting rather than from one isolated cause.

A weak procedure may combine with poor training, time pressure, ambiguous responsibility, inadequate information, a technology problem, and a missing safeguard. Removing any one of those conditions might have prevented the final outcome.

WIN therefore looks beyond the last visible error and examines the sequence of conditions that allowed the failure to occur.

This is consistent with WIN’s Real Root Cause approach: the most visible failure is not necessarily the deepest or most useful explanation.

Do Not Confuse Explanation With Excuse

Understanding why something failed does not mean declaring the result acceptable.

Explanation and accountability serve different purposes. Accountability addresses responsibility and appropriate consequences. Explanation identifies the mechanisms and conditions that produced the outcome.

Effective correction often requires both.

A system that refuses to investigate causes because responsibility has already been assigned may punish an individual while leaving the underlying failure mechanism intact.

Preserve Contradictory Evidence

Evidence that challenges an expected result is especially important to preserve.

Failed predictions, unexpected observations, negative results, participant complaints, anomalous measurements, implementation problems, and competing explanations can become more valuable over time as additional evidence accumulates.

WIN does not treat contradictory evidence as material to be discarded merely because it complicates a preferred explanation.

Preserve Institutional Memory

Organizations often repeat old mistakes because the people who learned from them leave, records are incomplete, unsuccessful projects disappear from view, or later decision-makers remember the conclusion without the reasoning behind it.

Important failures therefore require enough context for future reviewers to reconstruct what was attempted: the assumptions, evidence, expected results, implementation conditions, actual outcomes, competing explanations, corrections, and subsequent revisions.

The purpose is not to create a permanent record of embarrassment. It is to prevent valuable knowledge from disappearing and to make repeated failure less likely.

Avoid Learning the Wrong Lesson

Failure does not automatically tell an organization what to do next.

A failed intervention may tempt people to conclude that the entire objective is impossible when only one approach failed. Conversely, a partial success may encourage expansion even though the underlying mechanism remains poorly understood.

WIN therefore separates the observation that something failed from the explanation for why it failed and the decision about what should happen next.

Each requires evidence.

Correct Before Expanding

Discovering a weakness during a small pilot is less costly than reproducing the same weakness throughout a larger system.

When a consequential problem is identified, WIN’s development process allows time for diagnosis, correction, and additional testing before expansion.

A schedule, previous investment, public announcement, or desire to demonstrate progress does not make an unresolved weakness disappear.

Expansion is appropriate when the underlying capability and safeguards are sufficiently ready—not merely because the next stage was previously scheduled.

Retest the Correction

Making a change after a failure does not establish that the problem has been solved.

The correction itself must be examined.

Did the change address the actual failure mechanism? Did the expected result improve? Did the correction create new unintended consequences? Does the improvement persist? Does it work under relevant conditions beyond the specific case that produced the original failure?

Where practical, consequential corrections are tested before being treated as durable solutions.

Learn From Success Too

Success can also be misunderstood.

A favorable result may occur because an intervention worked as intended, because implementation was unusually strong, because circumstances were unusually favorable, because another factor produced the improvement, or simply because of variation or chance.

WIN therefore examines successful outcomes with some of the same questions applied to failures.

What appears to have produced the result? Can it be reproduced? Under what conditions? What limitations remain?

Understanding success is necessary before attempting to reproduce it at larger scale.

Failure Analysis Should Improve Future Decisions

The purpose of examining failure is not merely to produce a report.

Credible findings should influence the next decision. Depending on what the evidence shows, WIN may revise an explanation, modify educational material, change procedures, strengthen safeguards, improve measurements, redesign technology, alter training, conduct another pilot, delay expansion, or discontinue an approach.

Failure analysis has value when the resulting knowledge changes future behavior.

Turn Failure Into a Learning Loop

The useful sequence is not failure followed by a new unconnected attempt.

It is:

Expectation → Observation → Comparison → Explanation → Correction → Retesting → Updated Knowledge

Each cycle should leave the next cycle better informed.

That is how a research and development system improves without pretending to be infallible. Success provides evidence that an approach may be working. Failure provides evidence that something requires explanation. Unexpected outcomes provide evidence that the system may not yet understand an important variable.

What matters is whether the organization can use that information to become more accurate, more transparent, more correctable, and more capable of durable improvement.