Newborns' bilirubin levels are routinely monitored as a way of preventing possible brain damage... More >
What's Good Health Information?
The wide availability of health information — on television, in print, on websites and via search engines — brings with it a new set of challenges for consumers of medical and health information. Sometimes it seems as though we are bombarded with health information. A lot of it is hyped, taken out of context or boiled down to a sensational headline. The Internet— with its thousands of sites putting out information whose quality can be difficult to judge — brings its own new set of challenges for consumers of medical and health information.
How can a person know if the information they are getting is accurate?
Obviously, the source of the information is a big factor. The National Institutes of Health offer some tips for determining the quality of information on health and medical websites and a tutorial to help you evaluate medical websites.
But the source of information is not the only issue. Consumers need to be able to evaluate different kinds of information and understand how it was generated. Suppose you hear a reporter on the evening news saying, "A recent study has shown that exercise will add twenty years to your life." Such a statement should raise a few preliminary questions in your mind.
Unfortunately, the devil, as the saying goes, is often in the details. Perhaps the most poorly understood fact about scientific studies is that there are only a few specific types of research that are able to prove cause and effect relationships between one thing and another. Often, all a study is really reporting is that there is a relationship, or correlation, between one thing and another. Once again, take as an example our fictional exercise story. Let's say that the researchers in this study followed a group of retirees from age 65 to age 90. The subjects were asked to fill out questionnaires about how many hours per week they exercised; researchers monitored the retirees and discovered that those who said they exercised for one hour a day lived an average of 20 years longer.
So what is wrong with saying that this study proves that exercise makes you live longer? For one thing, if you look carefully, you will see that there is a possible bias in the selection of subjects; among other differences, people who retire are wealthier than average. Second, the researchers accepted self-reported data about how much subjects exercised; because people know that they are supposed to exercise, when reporting to a medical professional or researcher they might well exaggerate how much exercise they really did. Finally, because of the way it was designed, this study can, at best, do nothing more than establish an association between exercise and longevity. In other words, it shows that the two are related, but not how they are related. Because the study did not attempt to account for the myriad large and small differences between the individuals studied — from medical history to income to ethnicity to education and economic status, there is absolutely no way it could prove that a) exercise causes b) longer life. In fact, the findings of the study are perfectly consistent with other explanations, such as:
Medical and other academic journals in the sciences require that their authors explain the methodology they used. This is done so the methods themselves can be evaluated and so other researchers can try to test or replicate (reproduce) the findings. If other researchers perform similar experiments and cannot come up with similar findings, it is likely that the findings of the first study were affected by some factor other than the one being studied — anything from unaccounted-for qualities of the participants to an accidental flaw in the way the study was done. Suppose one study found that eating a grapefruit every day improved muscle tone in a group of elderly men, but other researchers were unable to replicate these findings. The likelihood is that some other, unidentified factor was the real cause for the muscle mass increase in the first study.
That issue, the fact that even a carefully designed, peer-reviewed "scientific study" — the kind quoted in news articles, on television and websites — can be wrong, is the other reason why it is important to know a little about some of the basic research designs used by medical and other researchers. The rest of this article considers some of the major types of studies and the strengths and weaknesses of each.
No comments have been made