The Fallacy of Bayesianism and the Popperian Path to Progress

In the landscape of modern physics and epistemology, there exists a profound tension between how we traditionally define knowledge and how scientific progress actually occurs. The prevailing view among physicists and academics is rooted in Bayesianism, yet this framework faces significant challenges when scrutinized against the reality of scientific discovery. By contrasting Bayesian probability with Popperian epistemology, we can better understand the nature of scientific theories and our evolutionary inclination toward flawed reasoning.

The Crucial Experiment and the Nature of Theory

Scientific progress is not merely a collection of data points; it is defined by the "crucial experiment." This is the "cherry on top of science," where two competing theories are tested against one another. If an experiment yields results that align with one theory while contradicting the other, the latter is ruled out. This process creates an open-ended quest for progress.

Currently, we possess only one theory for gravity: general relativity. Previously, we held two: Newtonian gravity and general relativity. Through crucial experiments, we have refined our understanding. However, this raises a fundamental point: science is not about reaching a "final theory." Instead, as the speaker notes, "all we have is better and better approximations to reality." Even general relativity, our current best model, is acknowledged as an incomplete description that will eventually require further correction.

The Bayesian Trap: Scientism vs. Epistemology

Despite the success of the Popperian approach, the "overwhelming majority of physicists are still Bayesian." This is largely due to academic tradition, where Bayes' theorem is taught as the standard for "intellectually rigorous" understanding. However, the speaker argues that this is merely a "species of scientism." While Bayes' theorem is a "perfectly acceptable statistical formula" for specific applications, it fails as an epistemology. It cannot guarantee the truth of a theory, nor can it provide the confidence that its proponents claim.

The primary failure of Bayesianism is highlighted by the history of Newtonian gravity. Prior to 1919, every experiment confirmed Newton's theory. A Bayesian would argue that, consequently, one should become "more and more confident in Newton's theory." This creates a logical paradox: the day before a theory is proven false is the day a Bayesian is most confident in its truth. This circular reasoning demonstrates why Bayesianism is insufficient for understanding the nature of scientific breakthroughs.

Popperian Realism: A Shift in Perspective

A "Popperian" approach resolves this paradox by asserting that at no point was Newton's theory strictly true. While it contained "some truth"—possessing a closer connection to reality than random conjecture—it was ultimately an approximation. Gravity varies as the inverse square law, but only approximately. Popperian epistemology allows for the rejection of false theories without the need to quantify confidence levels, acknowledging that our theories are always subject to correction.

The Evolutionary Hardwiring of Bayesianism

Why do we default to Bayesianism if it is philosophically flawed? The answer lies in our biological heritage. We are "evolutionarily hardwired for Bayesianism." Animals that cannot form "good explanations" rely entirely on Bayesian inference—observing repeated events, such as the sun rising or the heat of an object, to predict future outcomes. This is a survival mechanism for the biological world.

Humans, however, possess a "neocortex that can form good explanations" and "explain the seen in terms of the unseen." This provides a higher level of reasoning, but it is not instinctual. It "requires effort" and "deep thinking." We default to Bayesianism because it is the path of least resistance, mirroring the natural world at a purely biological level. Overcoming this default is the central challenge of human intellectual progress; we must move beyond the simple correlation of past events and embrace the capacity to construct deep, explanatory theories about the nature of reality.