Between the Lines: Finding the Truth in Medical Literature
  • Home
  • Latest Posts
  • Contents & Excerpts
  • Reviews
  • Events, Appearances & Press Coverage
  • FAQ
  • Contact & Press

Table of Contents


Introduction


Part One:   Context

Chapter 1 . The Importance of Denominators

Chapter 2. When Do Diagnostic Tests Save Lives?

Chapter 3 . Beware of What Seems Too Good to Be True

Chapter 4 . Why Medical Testing Is Never a Simple Decision

Chapter 5 . Hierarchies and Causes

Chapter 6. Assume a Spherical Cow

Chapter 7 . Why Some Blockbuster Drugs Don't Work for Everyone

Chapter 8 . When the Brain Gets in the Way, Part 1

Chapter 9 . When the Brain Gets in the Way, Part 2

Chapter 10 . The Beautiful Uncertainty of Science

Chapter 11 . In Praise of Not Knowing

Chapter 12 . A Constructive Way to Deal with Uncertainty



Part Two:   Evaluation

Chapter 13 . The Study Question

Chapter 14 . Study Design—General Scheme

Chapter 15. Observational Studies

Chapter 16. Interventional Studies

Chapter 17. Threats to Validity: Bias, Misclassification, and Generalizability

Chapter 18 . Statistical Analyses: Measures of Central Tendency

Chapter 19 . Hypothesis Testing and Measures of Association

Chapter 20 . Statistical Significance, Type I and Type II Errors

Chapter 21 . Threats to Validity: Confounding and Interaction

Chapter 22 . Study Results and Conclusions

Coda. Only the Beginning

Index

Introduction

Thick March snow is piling up outside my office window while I am hard at work putting finishing touches on this manuscript. I take a break to click over to the Table of Contents in today’s issue of the New England Journal of Medicine , and almost immediately I am captured by the intriguing title of a Perspectives piece: “What's the Alternative? The Worldwide Web of Integrative Medicine.”[1] I read it, and I am amazed at how pertinent it is to the overarching goal of this volume.

In her piece, Srivastava, who is an oncologist from Australia, weaves a sorry tale of a young mother who, through a series of events, came to be screened by an unethical quack for circulating tumor cells in her blood. The test turned up positive; she was told that she had advanced lung cancer and was given intravenous vitamin C at a cost of $6,000. Eventually she ended up in the article author’s office, where no evidence of cancer could be found. The author seems to suggest that policy measures to regulate what she refers to as “alternative medicine” would be the solution to this conundrum. But is that really so? Has regulation of the medical field assured integrity and patient safety? Let’s examine the evidence.

Medicine is one of the most heavily regulated fields in the United States. Yet in 1999 the Institute of Medicine’s report To Err Is Human uncovered nearly 100,000 deaths caused by errors in U.S. hospitals annually. A few years later, we heard that infections contracted in hospitals cause nearly as many deaths. At the same time the volume of discussion about the demand for overdiagnosis and overtreatment and their perils has been getting higher. While regulation can mitigate some of these harms, it cannot extinguish them completely. The real answer lies in education and empowerment.

The fury of the recent controversies surrounding screening for such diseases as breast and prostate cancers underscores the feelings of betrayal that the public feels when uncertainty is injected into dearly held truths. It reflects the failure of many people to understand, at least in some minimal way, the philosophical underpinnings and the method of science. Every scientific advance brings with it many more questions than answers; and as these questions are answered, new questions and new understandings arise. Science is fluid and uncertain and beautiful that way, and it should make all of us constantly question all of our assumptions.

But science can also seem intimidating and daunting and inaccessible. And as news outlets continue to pump out breathless reports of medical breakthroughs that make it look as if immortality were just around the corner, both lay people and those trained in the clinical sciences are put off by the escalating complexity of medical literature and leave truth finding to others. Such abdication is fraught with consequences, since those who are charged with interpretation and decision making are, on the one hand, not always equipped to do it and, on the other, view the task through the prisms of their own values and experiences. And, yes, clinical science is sufficiently imprecise to lend itself to such varied conclusions. How often have you heard the warning “Buyer beware”?

I have been sailing the seas of evidence-based medicine for nearly two decades, teaching methods of evaluation, serving as a peer reviewer and an editorial board member for several medical journals, and most recently blogging. All these years and activities have led me to believe that we can do much better: We can do better research, we can be better peer reviewers and editors, and we can be better readers and interpreters of the data. In this era of information overload, the incredibly shrinking medical appointment, and the emergence of the e-patient movement, being well-versed in the methods is no longer optional.

This book is here to help. It is intended to give you some basic tools with which to examine medical literature critically and systematically, to develop your own interpretation toolbox instead of relying solely on that of others. This is a toolbox that you can open every time you are faced with making a medical decision, so that, unlike the young mother in the New England Journal article, you can guard against charlatans of all stripes.

Who should read this book? Who shouldn’t? The current model of disseminating medical information relies on a layer of translators—journalists and clinicians. But I believe that every educated person must at the very least understand how these interpreters of medical knowledge examine, or should examine, it to arrive at the conclusions that ultimately make their way to the public. At the same time, both journalists and clinicians may benefit from this refresher. Based on news releases that I have seen describing studies , this volume is a must-read for the public relations professionals who develop those releases. Medical librarians may find this book particularly helpful: Being at the forefront of evidence dissemination, they can lead the charge of separating credible science from rubbish. Similarly, biopharmaceutical professionals within Medical Affairs groups, specifically those in the Medical Science Liaison roles, have found the ideas in this book helpful when I have presented them in workshops and seminars. Finally, one of my professional goals is to help improve the surprisingly variable quality of peer review in clinical literature. So, I particularly encourage new peer reviewers for clinical journals to read on.

A few words on the format of the book: I have purposely kept my tone conversational and references to a minimum, since most of the ideas that will be discussed are well accepted. When citations are unavoidable, I have made them fairly unobtrusive by putting them in the footnotes. I have also tried to keep computations low-key, focusing instead on conveying broad concepts. Furthermore, I have tried not to get into minutia or to explain everything to its limit, as fuller explanations, when needed, are easily accessed in more comprehensive texts. For these reasons, I am not expecting that this work will replace a course on how research is carried out and evaluated. But I do hope to make the reader aware of what a critical review of a paper looks like.

The book consists of two parts. Part One serves to set the stage for how we should think about science, particularly the science that informs the practice of clinical medicine. It is a compilation of short essays on some general concepts that interfere with the process of posing and giving rational answers to questions in medicine. Here we will touch on how shortcuts in thinking and cognitive biases make us prone to misinterpreting data. We will also make a brief detour into some philosophical issues that define scientific method, particularly as applied to clinical research. And most important, we will take a close look at uncertainty as the only certain feature of science and learn how to get comfortable with it.

Part Two is concerned with the fundamentals of evaluation, the tools that should be in the toolbox of everyone who wants to know what to make of a finding. We will examine ways to evaluate the components of a study: question, design, analyses, results, reporting, and conclusions.

For this journey, we will look at different studies to examine many different types of study designs. I have, however, chosen the landmark Women’s Health Initiative study that reported on the risks and benefits of menopausal hormone replacement therapy as a recurring guide throughout this discussion.[2] This study is available on the JAMA web site for free (see footnote for the link), and you can access it for yourself, should you want to navigate along with me. I will refer to it as “WHI-HRT” when discussing certain general points of medical literature, as well as in some specific instances when dealing with evidence.

Some of the things I say may seem contradictory at first: I may in one sentence extol the virtues of a randomized controlled trial and in the next knock it down. I may then proceed to tell you why observational studies lack certain elements of validity and shortly thereafter hold them up as an example of good research. There is a simple reason for this apparent contradiction: No study is perfect, all designs have strengths and weaknesses, and there is just no shortcut to knowing how to recognize them.

And that is the whole point of this book. I hope that reading and using it prevents you from falling prey to any kind of predatory quackery, be it “alternative medicine” or overly zealous disease mongering by the medical establishment. If there is only one thing you take away from this volume, it should be this: Every decision you make comes with risks and benefits. Do not let anyone with a vested interest pull wool over your eyes and blind you to these risks.

And one last thing: When I was growing up, I was taught that it was disrespectful to write in books. Well, I encourage you to write, underline, highlight, dog-ear, and leave sticky notes in this book (provided it belongs to you). I cannot imagine a more eloquent show of respect.

[1] Srivastava, R. What's the Alternative? The Worldwide Web of Integrative Medicine. New England Journal of Medicine 2012; 366:783–785

[2] Writing group for the Women’s Health Initiative Investigators. Risks and Benefits of Estrogen Plus Progestin in Healthy Postmenopausal Women: Principal Results from the Women’s Health Initiative Randomized Controlled Trial. JAMA 2002; 288:321–333 Available at http://jama.ama-assn.org/content/288/3/321.full.pdf+html

Web Hosting by Bluehost