Transcript

We’re moving down our list, introducing each of the five units in our proposed curriculum for science literacy.

This unit is the Methods of Science.

Right away some of you watching are asking how the methods of science differs from the logic of science.

If you’ve studied a little philosophy of science it’s even more likely to confuse you because there’s a long tradition in the philosophy of science of treating the logic of science as a synonym for the methods of science. The methods that scientists use to answer questions is just another way of talking about the logic of scientific reasoning.

So why do we need to distinguish logic and methods?

The reason why the distinction is important is because the way that philosophers of science talk about the logic of science is often very different from the way that scientists talk about scientific methodology, and how scientists are exposed to these questions in their education as scientists.

The Logic of Science

When we talk about the logic of science, and explore various models for how this logic works, we’re really asking about how different sorts of scientific claims can be rationally justified.

And that question gets translated into "what are the criteria for a good argument, where the conclusion is a scientific knowledge claim?".

These arguments are going to have some common features. The premises, for instance, will normally make claims about observable phenomena, what we can perceive or measure. They won’t all be premises like this, but some of them will be.

The conclusion, ultimately, will make an assertion that goes beyond what is asserted in the premises, and will often go beyond what is empirically observable at all.

These conclusions will have different forms, and they express different kinds of scientific knowledge claims.

One kind is a generalization, a statement of the form “All A are B”. All bodies move in such-and-such a way, all chemical reactions work like this, all metals have the following properties, all plants have the following properties, and so on.

Statements like this often express predictive regularities at the level of observable phenomena, but in generalizing they go beyond what is directly observable, by asserting something about all entities or processes within a particular category, even the ones that we haven’t observed and may never observe.

Other kinds of scientific claims that go beyond the empirical data include

• claims about historical events (what actually happened, at some point in time that we cannot go back and observe directly)
• claims about causes (we may observe that two variables, A and B, are correlated, but to say that A is the cause of B is to go beyond the mere correlation)
• claims about theoretical entities (these are entities whose existence we posit as part of our scientific explanations of the world, but which may never be observable in any direct sense).
  
  • Often these are part of theoretical models that we confirm or falsify indirectly by examining their testable consequences.
• predictions of future behaviors or future events

Altogether, claims like these are what constitute scientific knowledge.

Ultimately, the logic of science is the logic of arguments for various kinds of scientific knowledge.

Now, philosophers of science have been very good at unpacking the logical structure of these kinds of arguments, and there’s a whole industry devoted to questions and debates about what the criteria are for a good argument within each of these categories.

But it might be surprising to learn that scientists are largely ignorant of these discussions about the logic of science, when they’re framed in the language of logic and argument analysis.

What Scientists Are Taught About the Methodology of Science

What scientists are taught, in their education as scientists, are various tools for answering questions and solving problems, that are often quite specific to the scientific field in which they’ve being trained. What they come to know about scientific reasoning is embedded in their experience of using these problem solving techniques.

So, in physics and engineering, for example, they’ll learn a lot about error analysis, different methods for fitting curves to data points, linear regression, and so on. Social scientists will learn a lot about how to write surveys and analyze survey data, that physical scientists never have to learn. Everyone learns something about hypothesis testing in statistics classes, but the parts that are relevant to testing hypotheses in physics can be quite different from the parts that are relevant to testing hypotheses in psychology. In some fields you learn a lot about how to use data to select between competing theoretical models, in other fields these methods never come up.

And then there is practical instruction in applied research methods, which can be very specific to different fields. Learning how to identify and minimize errors in a large experimental physics lab is very different from learning how to identify and minimize errors in a psychology experiment where you’re interviewing subjects, or in medical research where you’re testing the effectiveness of a new drug.

Physical scientists don’t have to worry about the placebo effect, for example, but learning how to design studies that isolate and screen off the placebo effect is a very big deal in medical research.

Now, all of this is important for understanding how science works, so we need to talk about it. But we need to understand the differences in this kind of talk from what philosophers of science are doing when they talk about the logic of science.

Logic of Science vs Methods of Science

Here’s one way to conceptualize the difference between what I’m calling the logic of science, and the methods of science.

The logic of science is primarily about the justification of scientific arguments of different types, considered from the perspective of logic and argument analysis and the standards we use to judge reasons for believing anything. Because it’s couched in the language of logic and argumentation, it’s going to be very general and abstract, because it’s meant to capture the reasoning behind whole categories of scientific inferences.

The methods of science, on the other hand, are primarily about techniques for minimizing or neutralizing the errors in judgment and reasoning that human beings are naturally prone to commit, when we go about trying to answer specific scientific questions. Some of these techniques can be quite general in their application, but many will be specific to particular scientific questions.

Now, when I talk about techniques for minimizing errors in judgment and reasoning, I’m appealing to a tradition of thinking about science that was wonderfully captured by the physicist Richard Feynman. Feynman said that “science is what we do to keep us from lying to ourselves”.

I offer this as a slogan that captures the essence of what I’m talking about when I talk about the methods of science.

The background context is that science is a social practice conducted by real human beings, and human beings are not naturally equipped to do science well.

Our remarkable brains are great at detecting patterns, which is essential for science, but we’re also prone to seeing patterns that aren’t really there, and attributing causal significance to patterns that may be purely random. We’re subject to confirmation bias and dozens of other cognitive biases that are natural for our species but that make us prone to error in systematic and predictable ways.

A very fruitful way to think about scientific methodology, I submit, is to think of it as a set of protocols whose primary aim is to minimize or neutralize the distorting effects of cognitive biases on the practice of scientific inquiry.

These protocols are debiasing tools — they help to make our judgments less distorted, more objective, more reliable, more accurate, than they otherwise would be if left to our own devices.

This is the best way to explain, for example, why controlled studies are important for science, and why randomized controlled studies are superior to non-randomized studies, and blind, randomized controlled studies are even better, and why double-blind and sometimes triple-blind, randomized controlled studies may be necessary to screen off certain kinds of biases and errors.

This the best way to explain why certain data analysis and hypothesis testing techniques are used — because they force us to consider the totality of the data, rather than just the portions that our brains are naturally disposed to focus on.

This is the best way to explain why replication is important in science, and how confidence in a result can build over time as studies and experiments from different labs converge on the same result.

This is the best way to explain why peer review is important, and how the social organization of science provides both incentives and a mechanism for identifying and correcting mistakes.

What Kind of Understanding of Scientific Methods is Important for Science Literacy?

These protocols for minimizing error, these methods for doing science, will vary from field to field, and there are a lot of them. The closer you zoom in on any particular scientific field, the more of them you see. Part of the reason why it takes a long time to train scientists to do research in a given field is that these methods require understanding and experience and skill to apply properly. At this level it’s like learning a skilled trade or craft, and you need to be mentored into the field over a period of time to acquire the skill and judgment to use them effectively.

So we can’t expect physicists to learn the protocols for doing field studies in ecology, or survey methods in the social sciences.

Nor can we expect ordinary citizens to learn about all of these methods. They’re too many, they’re too diverse, and some of them require specialized training just to understand why they’re necessary.

But what we can expect, and ought to expect, is that any scientifically literate person be able to articulate in general terms the reasons why such protocols are needed, what functions they serve, and the role they play in justifying the reliability and authority of scientific claims.

That is something we can do, and it is something that science literacy should aim for.

And that’ll be the focus of this unit on the methods of science. Students should walk away with a richer understanding of the kinds of errors that human beings are prone to when conducting scientific research, and how different scientific protocols function to neutralize or mitigate those errors.

In the next unit in our proposed curriculum for science literacy, we’re going to talk about the "Landscape of Science".