31 No. 3
Scientific Method: Can It Help Promote the Public Appreciation of Science?
by David A. Evans
At a Committee on Chemistry and Industry (COCI) meeting in 2007 attended by then-President Bryan Henry, I flippantly stated that the principles of scientific method could be used as a blueprint for living one’s life. Henry immediately contradicted this comment and pointed out that the method had no relevance to emotional issues, such as love. Of course, Henry was right, but I still contend that many aspects of day-to-day life would be better explained and understood if scientific method were applied. This article illustrates some examples.
Both the Committee on Chemistry Education (CCE) and COCI have programs dedicated to the public appreciation of chemistry. CCE and COCI are notably diverse in the range of countries they represents. One lesson that we have learned is that public appreciation of chemistry varies greatly from country to country, and it is clear that tailored local initiatives are usually much more appropriate than generalized global programs. However, some issues warrant a universal approach, and scientific method is one of them.
What Is Scientific Method?
No singular definition of “scientific method” exists. A simple Google search provides numerous populist articles that reveal simple definitions of the type shown below, taken from Science Buddies1:
The scientific method is a way to ask and answer scientific questions by making observations and doing experiments.
In short, this is simply the way in which good science is properly conducted. A Google search of “scientific method steps” provides another delightful list of articles, aimed at all ages and levels of sophistication, which includes pictograms, songs, and mnemonics to help remember the steps. The following provides a fair summary:
1.1 Make observations
1.2 Relate causes to effects via propositions
1.3 Test propositions experimentally
1.4 Analyze the data and draw conclusions
1.5 Publish results following peer review
During this journey, the status of a proposition might move to a hypothesis to a theory and even to a law, but facts are elusive and only arrive when all attempts to disprove them are defeated. Chapter 3 of The Complete Idiot’s Guide to Chemistry2 provides a simple description of what can be involved in the various steps. However, the purpose of this article is not to critique the descriptions of these steps, but rather to indicate useful ways to promote the public’s appreciation of the method.
1.1. Make observations: In response to an observed effect, it is tempting to jump to a conclusion regarding the cause based upon our past experience, values, and prejudice. In a world where many people wear their ignorance of science as chic badges of honor, we are often faced with dialogue based on beliefs rather than evidence. Perhaps the most important first question for everyone—sadly, frequently ignored—is whether an observation is a one-off or can be repeated.
1.2. Relate causes to effects via propositions: As a hypothetical example, imagine that your headache disappeared after drinking an infusion of daffodil petals; therefore, it is obvious to you that daffodils cure headaches. But headaches disappear over time (this is often called “regression to the norm”) or for a variety of reasons (e.g., drinking water), so you are simply indulging your beliefs if you insist that daffodils cure headaches. Nothing is wrong with holding certain beliefs, but if you stop at this stage, that is all you have!
A very important lesson here is that association is not equivalent to causation. A good example of this lesson comes from the observation in a UK town that the incidence of child asthma had doubled in a period when the number of motor cars registered had also increased similarly. One feasible proposition is that car exhaust fumes were responsible for the increase in asthma. However, it was observed that the number of Asian restaurants in the town had also doubled in that period! In addition, it was pointed out that the south island of New Zealand had a high rate of child asthma, despite having very low traffic levels and lots of fresh air. The lesson to be learned is that data based upon good science is essential to determine causation. Furthermore, even if it were shown that car exhaust fumes were responsible, it would be vital to know the extent to which the effects were due to gaseous or particulate emissions in order to implement the correct engineering solution. Good science, technology, and engineering can solve problems. Traveling in hope based upon beliefs is usually a waste of time and resources.
1.3. Test propositions experimentally: Here, it is important not only to design experiments to test your hypothesis but also to consider alternative and null hypotheses as part of the process. A crucial factor is the inclusion of appropriate control experiments—failure to include these invalidates the whole study. A populist description of the intricacies of control experiments in clinical work is provided by Ben Goldacre in his recent book, Bad Science.3 This excellent volume, written by a practicing physician, provides compelling examples of the pitfalls caused by failure to follow scientific method. It is clear that proper design and execution of experiments is the heart of the process. A section about the placebo effect and how to include it in experimental design is particularly informative3 and illustrates the importance of the inclusion of control experiments. People outside this field would have been skeptical if informed that the number and color of sugar pills used as placebos have different positive effects on patients—but they do!
1.4. Analyze the data and draw conclusions: This is also a vital step—whereas the data are the ultimate product of the experimentation, their treatment and interpretation are open to bias and misrepresentation. This misrepresentation underpins the necessity for full publication of the raw data, discussed in the next section. In Bad Science.3 in addition to a chapter on the misuse of statistics, there is discussion of numerous instances of biased interpretation—some unconscious, and others willful. The dangers of “cherry picking” from within the data and from differing statistical treatments of outliers can lead to selective interpretation. Furthermore, the consideration of your data as part of a wider dataset published elsewhere is always wise. In clinical studies, a “Cochrane review”4 represents a pooling of data from similar trials to improve the accuracy of conclusions and to help deal with outliers.
At this point, you will have generated consistent evidence to support or rebut your hypothesis. In some branches of science, and particularly in chemistry, further research to relate causes to effects by establishing the mechanism will add significant value.
1.5. Publish results following peer review: The most important features here are the provision of all the data on which conclusions are based and the description of the arguments that are used to derive these conclusions. Peer review should not be claimed as an infallible process that guarantees the results, but it does provide a vital check of the validity of the experimental methods and the conclusions drawn. Peer reviewers cannot be expected to repeat the experiments, but subsequent full publication does allow anyone to check the data for themselves by using the methods described in the paper. Good data, widely communicated, are the bedrock on which future scientific endeavor is based.
The Public, the Media, and Scientific Method—A Personal Perspective
The principles of scientific method have relevance to members of the public who are not scientists. The problem is to introduce the public to the method in a way that is neither patronizing nor antagonistic to their beliefs, many of which reflect their experiences and lifestyles. Much of the information on which they base their opinions and beliefs will be derived from the media and press. In the UK, we are blessed with many newspapers that have almost zero scientific competence and that rely upon sensationalism to sell their copy—and a solid, good science story does not appeal. Several first-rate science correspondents are available, but they are bypassed for sensationalist scoops. To exacerbate matters, single-issue pressure groups and nongovernmental organizations (NGOs) jump on exaggerated misinformation to support dogmatic viewpoints. As in many countries, the utterances of vacuous celebrities rank much higher with the public than those of expert scientists and clinicians. Many of these, published in glossy magazines or autobiographies, are based upon the celebrities’ singular personal experiences. Again, Bad Science3 contains breathtaking examples of the ways in which pseudoscience and dogma are celebrated in the media in spite of the total lack of evidence for the benefits claimed. And we should be very worried—it was Jonathan Swift who observed, “You cannot reason a person out of a position he did not reason himself into in the first place.”
. . . the utterances of vacuous celebrities rank much higher with the public than those of expert scientists and clinicians.
So what can promotion of the virtues of scientific method contribute to the public debate? It is my belief that many of the large, well-organized initiatives that have been produced have created limited impact precisely because of their grand scale. Furthermore, it is arrogant of scientists to believe that the public has a duty to understand our viewpoint, and it is unrealistic to believe that we could educate the public to the point of meaningful understanding of a large number of complex issues. Education is surely a task that must be tackled incrementally—but on a broad front.
Some of my very modest personal experiences with promoting the virtues of scientific method follow in the paragraphs below.
The easiest place to start is with your nonscientist friends. Numerous sources referenced in this article provide fascinating, and sometimes amusing, stories in which the failure to employ good science produces nonsensical claims. Many of these fallacies are topical and include such subjects as detoxification, nutritional claims, and diets. I have committed these stories to memory and often introduce them in conversation with nonscientists. In the UK, we are fortunate to have pubs in which we can informally meet friends and acquaintances, and this atmosphere provides fruitful educational opportunities. In fact, this approach frequently leads to animated debates! I am becoming conscious of the pitfalls of preaching, but I do not believe that I have lost any friends—yet.
In response to a conversation in which someone presents an unsubstantiated belief as fact, I politely ask whether any data support the fascinating claim. The responses to this perceived challenge vary from stunned silence to constructive debate, to spluttering indignation. Accordingly, this request for data can be a dangerous practice, putting further invitations to supper in jeopardy.
Letters to newspapers that correct unscientific statements are published surprisingly frequently, especially in the local media. Ben Goldacre3 has strong advice for individuals: “To academics, and scientists of all shades I would say this: you cannot ever possibly prevent newspapers from printing nonsense, but you can add your own sense into the mix. E-mail the features desk . . . and offer them a piece on something interesting from your field. They’ll turn you down. Try again.”
Goldacre also encourages scientists to start blogs—his can be found at www.badscience.net—an opportunity for some of us to enter the modern world! Indeed, the Internet offers limitless opportunities for communication ranging from blogs and science chat sites to published articles—and the audience is potentially in the millions.
Contributions by Learned Societies
The Royal Society of Chemistry, justly irritated by advertisements for products that claim to be “chemical-free,” has offered a large cash prize for anyone who can present the society with a material that does not contain a single chemical—knowing that its money is safe! Many of our learned societies have similarly accepted the challenge of providing better outreach to the public, and they can contribute much more. I offer the following suggestions:
- Enhance the media centers of learned societies to develop improved relationships with the media via assistance with material and copy while helping rebut scientific nonsense at its source.
- Criticize advice or legislation in which lack of adherence to the principles of scientific method could produce negative consequences.
- Produce authoritative independent position papers on hot topics, written in jargon-free script.
- Ensure that web searches on science topics initiated by members of the public favor articles produced by learned societies. In my experience, Wikipedia—which can be edited by anyone—is often one of the top sites listed in response to a general query, whereas a learned society reference rarely makes the first results pages for a Google search.
We need to recognize that we can learn much from the NGOs whose communication skills and media awareness are awesomely developed. Perhaps the best way forward is for learned societies to become accredited as NGOs themselves and thereby gain seats around the tables of relevant international organizations, such as the United Nations Environment Programme and the World Health Organization. It is pleasing to note that IUPAC is already taking positive steps in this direction.
Finally, Peter Mahaffy and his colleagues in CCE, in their review entitled “Chemists’ Understanding of the Public,”5 conclude that one of the most important contributions for IUPAC would be to provide assistance to chemists in honing their public communication skills. One component of this movement would be to help ensure that all forms of communication should always pay heed to the principles of scientific method and should clearly enunciate the data on which the communications are based and the arguments used to draw conclusions. As such, this is akin to a teaching seminar—which is exactly what it needs to be!
- Science Buddies, www.sciencebuddies.org.
- Ian Guch, The Complete Idiot’s Guide to Chemistry, Alpha Books, New York, 2006.
- Ben Goldacre, Bad Science, Fourth Estate, London, 2008.
- The Cochrane Library, www.cochrane.org, published and hosted by Wiley InterScience.
- Peter Mahaffy, “Chemists’ Understanding of the Public,” July-Aug 2006 Chemistry International, p. 14 (www.iupac.org/publications/ci/2006/2804/4_Mahaffy.html).
David A. Evans <email@example.com> is a member of the IUPAC Committee on Chemistry and Industry (COCI). Evans’s interest in public appreciation of science predates his retirement as head of Research & Technology at Syngenta.
last modified 23 April 2009.
Copyright © 2003-2009 International Union of Pure and Applied Chemistry.
Questions regarding the website, please contact firstname.lastname@example.org