Sensitive issues
I reviewed a manuscript recently. Researchers were reporting a new test to diagnose `Christmas pudding embolism’ (details altered to preserve confidentiality).
Using a `mistletoe test’ (well, the baseline investigation they chose was not that well known either!) you achieved a sensitivity of 15% (95% CI 10 to 23%), and the new `red-nose reindeer’ test achieved a sensitivity of 49% (95% CI 45 to 55%). They concluded that the `red-nose reindeer’ test was at least 3 times more sensitive and should considered for routine diagnosis of `Christmas pudding embolism’.
Problem? Yes… a misconception that invalidates their conclusion.
Unlike most tests, sensitivity (and specificity) does not begin at 0%. A test is completely useless when the probability of getting a positive or negative result is no better than chance. If chance alone were in place, the probability of getting either a positive or negative result is … 50%. So, that’s where sensitivities and specificities start. Ergo, the `red-nose reindeer’ test is much worse than the `mistletoe test’ (it is much closer to 50%).
So how do you interpret a test result that has a sensitivity of 15% and a relatively narrow confidence interval?
Here comes the `science’ bits….
Unlike the usual interpretation, where in a test with high sensitivity, a negative result effectively rules out disease a `low’ sensitivity (i.e. way below 50%) has the interpretation that a negative result effectively rules in disease (ie a negative mistletoe test is a great way to diagnose `Christmas pudding embolism’). In practice, it is never used as such, but more likely to reflect the inappropriateness of the `mistletoe test’.
I guess the bottom line is no matter how confident you are about your research, it’s always a good idea to get a statistician to look through your work prior to submission.