Evaluating Medical Research

First published in The Lawyers Weekly August 11, 2000,  Vol. 20, No. 13

Personal Injury lawyers frequently face the daunting task of evaluating conflicting medical evidence - and few lawyers personally have the benefit of medical training or clinical experience.

Medical expertise has become more democratic - and more rapidly since the birth of Evidence Based Medicine in 1990.

In the past, counsel previously measured the reliability of medical opinion by the reputation and eminence of the medical specialist. Dr. National Authority was trusted to toe the official line of his specialty.

Now a specialist’s individual clinical experience is less important than his ability to interpret for the court the experience of his colleagues world-wide. Conclusions from a hundred personal cases of ruptured spleen are less reliable than the results of 10,000 cases reported in the medical research literature.

As the exponential growth of scientific knowledge strengthens the art of medicine, medical tradition and authority give way to testable truths.

Propositions of medical causation may be considered plausible because a model can be constructed on scientifically-verified principles. However, the acid test is whether there is statistical evidence of an association - "does it happen in practice?" There may be theoretical reasons for believing that cervical whiplash without a blow to the head could cause Mild Traumatic Brain Injury, but is there objective evidence that it actually occurs?

Particularly when the results of clinical studies are conflicting or counter-intuitive, even an approximate estimate of the strength of a research paper is valuable.

In general, the strongest studies are the rarest. The main types of clinical research in descending order of frequency - and increasing order of reliability - are case reports, case series, case-control studies, cohort studies, systematic reviews and meta-analyses.

Case reports are the weakest clinical evidence for causation. They may nevertheless be invaluable in alerting to hitherto unrecognised associations, as occurred with limb deformities and Thalidomide given to the mother during pregnancy.

However, one swallow does not a summer make. Case series give a more reliable indication of trends. If Fibromyalgia Syndrome occurs after 100 consecutive soft tissue injuries to the neck more often than after 100 consecutive leg fractures, the effect is more likely to be real than if it there are only half-a-dozen individual reports in the medical research literature.

Nevertheless, supposed causal connections may prove spurious on more critical analysis. Case-control attempts to remove recognised confounding variables by looking at statistically valid differences between a population of patients and a population of patients or healthy individuals pair-matched except for the feature being studied. The finding that most patients are unable to walk for 9 months after circumcision is less startling if, like them, uncircumcised subjects matched for age learn to crawl first.

Cohort removes further selection bias by following sufficiently large normal populations over time to track the appearance and progress of disease processes. Whether Low Back Pain is caused by a rear-ending MVA can be determined only by studying the spontaneous appearance and disappearance of the symptom in an otherwise healthy, uninjured population.

Huge population cohorts must be studied to determine whether major trauma precedes the onset of multiple sclerosis.

Systematic Reviews evaluate the strengths and weaknesses of all relevant research to date, and allocate a reliability score to the strength of evidence for a given proposition. Only 4 trials of bedrest for acute Low Back Pain or sciatica proved to be of high quality, and they showed little or no benefit compared with advice to stay active.

Meta-analysis involves the statistical manipulation of data from all available quality studies which contain sufficient detail. By combining the populations studied by each writer, spurious and anomalous findings are eliminated, and trends which are not statistically significant in an individual study may prove to be valid. Although individual studies give conflicting results, cognitive therapy in mild to moderate depressive disorder has superior effects compared with antidepressant medication.

However, the principle of "garbage in - garbage out" is not completely eliminated, even in meta-analysis.

All else being equal (which it never is), other properties of research findings will favour one result over a conflicting result. Are the findings replicable by different researchers in dissimilar surroundings? If a dramatic improvement in outcome of chronic pain is reported in only a handful out of all the clinics using a particular therapy, the defence cannot legitimately claim a better treated prognosis.

Was the article published in a journal whose original articles are peer-reviewed, that is, critiqued by a number of other research workers in the field? Whether or not the research paper in question was peer-reviewed, subsequently published comments from readers are an important part of the material to be assessed.

Thus, when faced with a bewildering array of medical research literature, the personal injury lawyer can quickly start to weight the probable reliability of the conflicting conclusions according to the type of research being reported.

Copyright © Handbook of Legal Medicine