Tom Treasure Weblog

Sometimes treatment effects are obvious. Ask someone who was unable to see due to a cataract and who has had successful surgery. Ask someone who suffered pain and immobility due to osteoarthritis of the hip and has had the joint expertly replaced. Their lameness and blindness were progressive and unremitting; the effect of treatment is evident and makes complete mechanistic sense. “The lame walked and the blind recovered their sight” - in former times only a miracle could achieve such relief (Matthew 15:29-31).

For other conditions treatment effects are not so obvious. Empyema is an example. It is caused by microbial infection and presents at different stages in its progression, with varying degrees of severity, and follows a variable course. Nature does not do a very good job when left to her own devices. Since the time of Hippocrates surgical drainage has been an option but modern management reserves surgery for unresponsive cases because most patients will get better with antibiotics and less invasive drainage procedures. For a summary see the chapter in “The Evidence for Cardiothoracic Surgery” (tfm publishing 2005). What I want to consider here is the belief that the instillation of a fibrinolytic (such as streptokinase or urokinase) has benefit.

The inflammatory response to infection results in deposition of fibrin in the pleural space and the pus becomes loculated and the lung trapped. The rationale is that the if the fibrin can be broken down or “lysed” by fibrinolytic enzymes such as streptokinase or urokinase, drainage should be more complete and the lung should re-inflate more fully and the natural processes will have a much better chance to complete healing. This treatment is complimentary to antibiotics and drainage and is less invasive and thus preferable to surgery. The question is: is it effective?

Five trials designed to study this question reported between 1997 and 2001. They included 24 to 53 patients each, an average of 41, small numbers to address such a question. A Cochrane meta-analysis found them to be underpowered, even when pooled, for the major effects. To resolve the uncertainty, Rob Davies and his group in Oxford, England set up a double blind randomised controlled trial (RCT) which reported on 454 patients, more than twice the number of patients in all the previous trials put together (NEJM 2005;352:865-74). The trial was meticulously carried out with careful attention to blinding and protocol adherence. There was no difference in any outcome, other than a small number harmed by streptokinase. In a trial of this size and breadth any missed treatment effect would have to be very small. Any subset that might have benefited would have to be balanced by one or more other subsets where the effect was in the opposite direction. This Oxford RCT is Level 1++ evidence (http://www.bhiva.org/pdf/pregex4.pdf). I thought that settled the matter but no - eminence struck back.

The BMJ published an editorial addressing the issue (BMJ 21st January 2006;332:133-4). The authors opine that Oxford RCT has major weaknesses while citing, without comment on quality, their own studies which are a fraction of the size. They cite one as showing urokinase and streptokinase as “equally effective” – they (and the BMJ editors) failed to mention that they might be equally ineffective for there were no untreated controls. Even in the title they seek to diminish the worth of the new RCT evidence with the quizzical subtitle “A step forward … ?” The piece has no specified research strategy and is Level 4 evidence at best, but the senior authors are known names in pleural disease, including Richard Light who has contributed so much on pleural disease including giving us Light’s criteria. Hence my phrase “Eminence strikes back”.

Another contradictory publication concerns the drug aprotinin, used to reduce bleeding associated with cardiac surgery. Researchers have been criticised for accumulating too much evidence (The Lancet 2005;366:107-8). There have been 64 RCTs and according to expertly performed meta-analysis, since the 12th the story has not changed: aprotinin reduces bleeding. The new issue that has arisen is about harm associated with its use. It is procoagulant and from its earliest phase II studies there was concern that a thrombotic effect might harm the brain, myocardium or kidneys (Ann.Thorac.Surg. 1993;55:971-6). Ten years later a meta-analysis of 35 placebo controlled trials in 3879 patients (Grade 1++ evidence) reassuringly showed no higher rates of renal failure, myocardial infarction, or death and there was a reduced risk of stroke (JTCVS 2004:128:442-8).

Dennis Mangano, a very well regarded cardiovascular anaesthesiologist has contradicted all of these conclusions (NEJM 26th January 2006;354:353-65). He dismisses the Grade 1++ evidence in a line by saying “nearly all investigations were sponsor-supported and therefore carried unavoidable bias”. If we applied that assumption generally it would leave a big hole in the evidence on heart failure, hypertension, angina, asthma, chemotherapy etc. He draws opposite conclusions based on propensity and multivariable analysis of observational data on 4374 patients divided into those treated with aprotinin, those treated with other antifibrinolytics (aminocaproic acid or tranexamic acid) or who received none of these drugs. The choice to treat patients with aprotinin is a conscious and deliberate medical decision reserved in most practices for the higher, indeed the highest risk patients, while those at lower risk are not treated. This is about as far from random allocation as you can get. We will address the statistical issues in more detail elsewhere but it seems to us likely that the analysis failed to correct for all the differences between these patients. This is Level 2- evidence at best. It seems to fly in the face of all received wisdom about statistical analysis, and to be improbable that mulitivariable analysis of observational data will reveal the truth while meta-analysis of 35 placebo controlled trials is wrong. Nevertheless NEJM published it. It looks to me as though it is another case of eminence trumping evidence.

It is not surprising that these eminent authors seek to defend their positions. They have written papers and books, and are opinion leaders and it must be hard to see their teaching challenged and disproved, but in the world in which I live, evidence for what we do matters. Effective medicine sits alongside ineffective medicine. When the circumstances are complex, we cannot tell which is which without careful sifting of evidence. Cause and effect, benefit and harm, are less obvious in treatment of empyema or managing coagulopathy around heart surgery, than fixing lameness and blindness.

While eminence defends its position, others try to seek improve the chances for evidence. In Paris last week (8th-10th February 2006) I was invited to join thirty others, from North America and Europe, to work on refinements of the CONSORT statement (Consolidated Standards of Reporting Trials - http://www.consort-statement.org/) to make them more applicable to surgical and other non-pharmacological research. The group included leading trial statisticians, clinicians and editors from JAMA, BMJ, Annals of Internal Medicine and The Lancet. Trials in surgery are not easy and many find it difficult to comply with CONSORT requirements (EJCTS 2004;25:299-303) but it is discouraging if the leading journals flout accepted standards of evidence in the debate on important issues such as bleeding after heart surgery and the management of empyema.

Posted by Tom Treasure at 06:58 AM | Permalink

On 7th December 2005 I had the privilege of being one of a dozen academics before whom Dr Özcan Birim stood to defend his PhD thesis. We Court of Examiners processed in our full regalia of academic gowns and hoods; exactly an hour later at the sound of a gong and the words “Hora est!” we processed out again to reach our verdict. In the intervening 60 minutes Birim introduced and faced questioning on the research contained in his thesis, a bound volume that will grace many libraries’ shelves. It contains eight first author papers, most in print, some in press, and other chapters giving introductory context and conclusions. In a steeply raked lecture theatre in Erasmus University in Rotterdam young Dr Birim stood before us, flanked by two young colleagues, all three in white tie and tails. The theatre was open to the public for the occasion and contained many members of the hospital, university and importantly, Özcan’s parents and family.

It caused me to reflect again (for I have done many times before) on the purpose behind a surgeon in training devoting time to research and publication, and in some instances, earning a further postgraduate qualification for research. Very few go on to become academics and many will never write a paper or do research again. Why do they do it? I can identify four quite separable objectives and to let you understand where this is all heading, I will state now that, in my view, the way they do it in Holland achieves all four - and they do it in style.

1. Appropriate personal development for the tyro surgeon.
2. A real contribution to knew knowledge.
3. A measure of achievement.
4. A rite of passage.

It is believed that a period in original research, as opposed taught courses, develops ability to appropriately modify practice in the light of new knowledge. It equips us for an unknown but inevitably changing future. Some still advocate time in the animal laboratory but in those far off days when it was the norm for every surgical department ot run an animal laboratory, biological knowledge and technical skills overlapped between animal and clinical experimentation. Birim’s work is in lung cancer. There is no animal model for its surgical management. His thesis comprises extensive literature reviews, statistical analysis of risk factors and comorbidity, life table, survival and meta-analysis. These are the vocabulary and skill of modern health care research and he has acquired them. This satisfies my first requirement – it is appropriate personal development.

To make an important contribution to knowledge (my second objective) would indeed be an achievement during a brief spell in a modern basic science laboratory. The individual’s contribution is likely to be a small part of a major project. On the other hand, performing clinical studies and undertaking secondary research (systematic reviewing and meta-analysis) cannot be done without the background knowledge that a clinician brings and it changes practice for the better immediately. Birim’s book adds to the knowledge we use to treat lung cancer. He has added more than he could have done in equivalent time with a young surgeon’s toolkit doing Western blotting in knock-out mice.

It is a competitive world. We have plenty of applicants competing for posts. We need objective ways for individuals to distinguish themselves. In Britain we still produce a thesis of our research work. John Kirklin used to decry the practice, arguing that peer reviewed papers were the proper currency of research endeavour. Birim has his first author peer reviewed papers published in international journals and he has them bound in a book. Two-for-one. How smart is that? My third objective achieved.

Finally, the rite of passage. In Britain we examine a thesis behind closed doors: a drab process which is all too often an anticlimax after much hard work. Defending your thesis in front of your parents, family, friends, compadres, and your surgical chiefs certainly qualifies as a rite of passage. It was great day for Özcan Birim.

Posted by Tom Treasure at 05:08 AM | Permalink