The structure of metascientific revolutions

“How do scientists proceed when aware only that something has gone fundamentally wrong at a level with which their training has not equipped them to deal?”

It’s an extraordinary time to be a scientist.

In The Structure of Scientific Revolutions, historian of science Thomas Kuhn gives an account of scientific progress. Real progress and scientific advancement, he argues, doesn’t happen through the gradual accumulation of new facts in the scientific record. Rather, real change in science happens only through crisis, when old paradigms about how the world works can no longer explain what scientists see.

As more anomalies that contradict our paradigm accumulate, a time of extraordinary science can begin, where previously unthinkable hypotheses and unimaginable experiments can be pursued. These moments of revolution allow for new breakthroughs and thus new paradigms to emerge. After this paradigm shift, a quieter period of “normal science” can gradually fill in the puzzles and gaps of a theory, until new anomalous results start to show up and the cycle begins again.

Science is in crisis. It’s not a crisis in just one particular discipline or theory, but a crisis in our metascientific paradigm, where the infrastructures and social contract of science are coming apart. This social contract for modern science, defined by Vannever Bush after World War II, holds that public investment in autonomous, self-governing institutions for basic scientific research will fill a reservoir of knowledge that can be tapped to translate into technological innovations that provide material benefit to a country and its people.

Today, we are faced with several overlapping breakdowns of this model: science is getting less bang for its buck, with more and more investment in the reservoir translating to less and less downstream impact. Our institutions are turgid. Our papers and findings don’t replicate. Nearly 80% of scientists would significantly change their research programs if they were less constrained by existing funding opportunities. Mistrust of science is growing. And that funding is now being cut to its lowest level in decades, throwing the whole system into perilous chaos.

These aren’t new concerns. From its origins, the Bush paradigm was challenged even by top scientists themselves for how it would make scientists unproductive with the bureaucracy it required—wasting time on committees, reviewing, and chasing grants (and therefore trends) rather than pursuing an actual frontier of research. Leo Szilard, the physicist and biologist who first conceptualized the nuclear chain reaction and helped launch molecular biology, warned in 1948 how a scientific system built on rewarding only certain kinds of output and metrics would be self-destructive in the long run: “the scientific workers in need of funds would concentrate on problems which were considered promising and were pretty certain to lead to publishable results. For a few years there might be a great increase in scientific output; but by going after the obvious, pretty soon science would dry out.”

The paradigm also faced challenges from outside the scientific ranks. As the Cold War began to cool off in the 1970s, US lawmakers began to question both the value of public investment into that reservoir of basic research and the autonomy of the basic research enterprise. Should we keep pouring money into research with questionable connections to the problems everyday Americans face? Republicans challenged NSF-funded programs teaching about human cultures and sexuality; Democrats challenged spending taxpayer dollars without direct democratic oversight. How should the government oversee the choices of what research should be funded or not?

Historian of Science Melinda Baldwin writes about these debates and how the arguments that played out during the resulting Congressional oversight hearings have continued to shape how we think about peer review, scientific autonomy, and government funding today. The members of Congress running these hearings argued that research funded by NSF should be more tangibly beneficial to the American public, with more oversight and transparency to the process of program selection and grant review, arguing that:

We are dealing here with a finite quantity of money taken in the form of taxes from people against their will in a very difficult time in our economy and being spent by a Federal agency, spent in a manner now questioned. [The NSF] must demonstrate that it is using tax money in a prudent manner, in such a way that the taxpayers can expect that there will be some payoff from NSF-funded research.

In Kuhn’s model of scientific crisis and paradigm shifts, the first anomalies and hints of a crisis rarely trigger full scale change; scientists might argue and debate, but often continue to reconfigure the dominant paradigm to accommodate the new realities. In response to the challenge from Congress, NSF leaders doubled down on defending the existing scientific paradigm, arguing that programs should continue to be chosen and overseen by scientist-program managers, and that grant review by an expert community of peers expecting anonymity and confidentiality—not democratic oversight from non-experts—was the only way to evaluate and support good science.

Ultimately, these Congressional hearings resulted in a downsizing in funding for the specific programs that Republicans objected to, while largely maintaining the status quo of how NSF program managers and peer reviewers chose which programs and proposals to fund. But they also fundamentally changed the role of peer review in the decades to come. In an attempt to protect the scientific autonomy of the granting process, the NSF scientists further separated the scientific community from its domain of societal impact and established peer review as the only legitimate process through which research can become “actual science.”

Baldwin’s paper shows how this belief has indeed had a pretty short history, as well as far reaching consequences. It is this entrenchment, she argues, that has sown the seeds of the contemporary crisis of peer review we’ve faced in the last decade. If peer review is so great, why are so many peer reviewed papers found to be incorrect or even fraudulent? How can a system designed to follow the crowd of existing experts find and reward something truly innovative? The expectations put on peer review as the sole arbiter of good and valuable science goes far beyond its original purpose and its capabilities, leaving both science and society disappointed in the results.

***

By the mid 1990s, the paradigm was under stress again. The more open-ended research in physics and chemistry that sparked booming industries of the mid 20th century was winding down. Bell labs was spun out after a decade of drastic funding cuts following the breakup of AT&T’s monopoly, a shadow of its former self, and the unfettered research that it represented was on the decline everywhere, with technology companies focused on short term research that would maintain the relentless march of Moore’s law.

And with the end of the Cold War, politicians were again asking whether the investment in basic research was worth the trade-off with more pressing domestic needs. In 1995, the chief of staff of the House Science Committee and a researcher in the Environmental and Societal Impacts Group at the National Center for Atmospheric Research wrote a policy article in Science about how the Vannevar Bush-era social contract of science was fundamentally changing:

A major ecological function of the social contract is to shape the expectations of both science and society. Science expects autonomy and support. Society expects substantial benefits based on the justifications scientists offer for federal support.

Changes in the ecology of science may render the contract unsustainable; with the Cold War ended, science is adapted to an obsolete environment. Other environmental changes include (i) a dissatisfied public ready to reduce the federal government's size and reach; (ii) deficit-reduction strains on funding, leading to many program reductions; (iii) increasing public awareness of problems that neither science nor government has resolved, including racism, drug abuse, breakdown of community, and crime; and (iv) two decades of decay in real wages, leading to politics focused on the grievances of the middle class.

While these political forces continued to demand oversight and tangible benefit, science had continued to pull away, culturally isolating behind peer review, paywalls, and the narrative of “pure” science and rationality. The authors argue that this isolationist tendency was built into the fabric of the Bush paradigm, and its entrenchment over the decades was a root cause to the current crisis.

They push for a renegotiated social contract, where science is more engaged in the problems that matter to people, and in a richer debate about the role of science as only part of the approach to addressing these problems. As Vannever Bush himself noted in The Endless Frontier, “Science, by itself, provides no panacea for individual, social, and economic ills…Science can be effective in the national welfare only as a member of a team.”

Under democratic accountability, science is consciously guided by society's goals rather than scientific serendipity. Good science is necessary but not sufficient; association with a societal goal is required. The Bush paradigm discourages explicit association with goals that are not those of science. Social accountability leaves to scientists a broad scope of scientific choice. Denial of accountability encourages elitist isolation.

Improved justifications will sustain political support for science because support is strengthened by performance commensurate with expectations, and expectations of science are a function of justifications made in the process of securing funding. By assuming the automatic generation of benefits, Bush's social contract precludes realistic expectations of science, implying that science can solve some problems that, in fact, alone it cannot. Reliance on an outdated social contract leads to a loss of faith in science and a subsequent loss of political support.

The opportunity for science to participate in such socially-impactful team projects and debates was everywhere in the 1990s. With emerging fights over GMOs and the food system, vaccines and the pharmaceutical industry, and climate change and fossil fuels, it was (and continues to be) a time of what philosopher of science Jerome Ravetz termed Post-Normal Science—times where policy decisions with high stakes need to be made in conditions of high uncertainty, where normal science (in the Kuhnian sense) isn’t enough to give a clear answer about what must be done. In these uncertain times, Ravetz argues, we have to consider the quality of research more holistically, not only through the lens of a small group of peer reviewers, but through involvement of other stakeholders with other understanding of the problems to be solved.

Scientists, on the other hand, tended to see these debates not as opportunities to meaningfully contribute to issues that mattered to people, but as attacks on science itself. The paradigm was under siege and had to be defended aggressively in turn. Academic conversations about metascientific questions of the social and cultural dynamics involved in funding, practicing, and communicating science, or about the political and economic questions involved in applying science in the real world became such lightning rods of conflict that the ensuing debates came to be known as the “Science Wars.”

***

Thirty years later, we often seem to be stuck with the same war mindset, with many of the same people making the same arguments for the same isolationist paradigm, no closer to addressing the same cascading set of crises. But while some still publish articles defending the ROI of basic science in paywalled journals and others go through the motions of calling for people to simply “trust the facts,” many others are taking up the call for extra-ordinary metascience.

More and more, there are new experiments for identifying problems, building teams, funding research collaborations, communicating and engaging in the world, and designing new socio-technical approaches and applications. Where anomalies and challenges to the paradigm previously caused scientists to pull further away, today many great scientists are actively looking for new paradigms and new ways of being scientists in the world. And where such experiments themselves and even research on the history and social science of metascientific questions themselves were once challenged as anti-science, today more and more scientists are calling for more study and more experimentation when it comes to peer review, and publication, among many other concerns. But as these experiments unfold, it’s critical that we don’t repeat the same mistakes of focusing internally on the mechanics of academic communication and further isolate science upstream and outside of societal impact.

We don’t know yet where all this experimentation will lead. The only certainty is that we can’t go back to what was previously “normal.” For Kuhn, it is this recognition that ultimately marks the transition to extraordinary science, when:

the anomaly itself now comes to be more generally recognized as such by the profession. More and more attention is devoted to it by more and more of the field’s most eminent men. If it still continues to resist, as it usually does not, many of them may come to view its resolution as the subject matter of their discipline. For them the field will no longer look quite the same as it had earlier.

What might this new science look like?