Strong Inference

Category: Research

source

Strong Inference

Certain systematic methods of scientific thinking may produce much more rapid progress than others.

What is the Problem?

The process of doing research has become less standardized in some fields, especially when compared to the more structured approach of fields like molecular bio and high-energy physics.

When a formal scientific method isn't adhered to, especially when formulating hypotheses, the process can become less efficient and less effective, leading to fewer discoveries and slower progress over time.

Summary

John R. Platt critiques the application of the scientific method in modern research, arguing that the process has become less structured and less effective in some fields. He proposes a more structured approach to hypothesis formulation, which he calls "strong inference".

The paper goes on quite a bit about historical examples of the principles of strong inference working in practice, as well as a detailed breakdown of how to systematically apply the method to research.

  1. Devise alternative hypotheses - Generate multiple competing hypotheses that could explain the phenomenon being studied.
  2. Devise a crucial experiment - Design an experiment that can unambiguously distinguish between the competing hypotheses, or at the very least, eliminate some or all of them.
  3. Carry out the experiment - Conduct the experiment and analyze the results.

Strong inference is essentially to carry out this process at every vertex of the logical tree of inquiry, and to do so in a systematic and structured way. He suggests keeping a notebook explicitly for this, and to pay particular attention to the process of hypothesis generation.

Key Insights

  • Have multiple competing hypotheses, and come up with the most efficient way to eliminate them via experimentation.
  • Systematically, explicitly, and regularly follow this process, from hypothesis generation to experimentation to analysis.
  • Ask yourself the two questions: "How would we know this hypothesis is wrong?" and "What hypothesis does this experiment disprove?"

Notable Design Details/Strengths

  • Deviations from strong inference only really manifest themselves as useless delays in the research process. Many scientists do a lot of busywork for no reason, which could be avoided had they spent more time formulating hypotheses.
  • Strong inference is strongly dependent on the actual induction being done when formulating hypotheses. This needs to be logically sound.
  • It is a system that works if done correctly, as it's essentially the minimum amount of work needed to make a discovery without just getting lucky.