Is osteopathy research relevant? A challenge has been made
Article Outline
As a profession working in an environment where healthcare policy is determined by a curious blend of science and politics, we can't escape numbers wrapped up in statistics. Many individual practitioners might be able to avoid statistics entirely, however, this is a luxury not afforded to those who represent the profession to third parties such as researchers, universities, health insurance companies, government agencies, and importantly, the media. One of the reasons we can't escape numbers is because they help summarise the effectiveness of our interventions. Effectiveness is important because, with few exceptions, consumers don't enjoy paying for healthcare services that fail to ‘work as advertised’.
A recent and prominent example of the importance of effectiveness data was the publication in April by Ernst and Canter of a systematic review of systematic reviews for spinal manipulation in the Journal of the Royal Society of Medicine,1 and the ensuing flurry of media attention the article attracted. The conclusion of the paper was that spinal manipulation is not effective for any condition, and that “spinal manipulation is not a recommendable treatment”. The main message of the media release was similarly blunt, and was widely publicised in UK press.
Various critiques of Ernst and Canter's article have been forthcoming and essentially highlight problems related to the methodology they employed to conduct their study; the limited operational definitions of manipulation; and sources of bias.2, 3, 4, 5 One of the specific criticisms levelled at the paper is that the review was focussed solely on studies of spinal manipulation, and that trials incorporating complex treatment packages were excluded.2 The argument is therefore developed that osteopaths rarely ever use spinal manipulation in isolation and so this review is not representative of osteopathic treatment and therefore does not represent a challenge to the relevance of osteopathy. Regardless of whether this point is true, some damage may have already been done. The review has already been published, and its conclusions have been widely publicised.
While there are problems with using systematic reviews to summarise treatment effectiveness, it may be prudent to consider the following: where are the research data from the osteopathic profession that demonstrates the effectiveness of the interventions commonly administered, some of which have been in use for more than a century? If the evidence was there in a format consistent with current standards in research reporting and biomedical publishing, then that evidence would be included in systematic reviews. If the research is there, but suffers from methodological weaknesses (such as poor operational definitions of the manipulative protocol), or flaws in reporting the data (such as failing to report drop outs) then we must resolve to improve research protocol design and reporting in order to ensure that it is not excluded from systematic reviews. If we don't enjoy being in the firing line, then we may also need to examine the unwitting contribution we have collectively made to the ammunition of critics (such as Ernst and Canter) by failing to adequately investigate and document the effectiveness of our treatment.
This call for clinical research may be downplayed on the basis that osteopathy “can't be summed up in a test tube or in a laboratory”, or “that a reductionist research paradigm cannot investigate a holistic patient centred treatment approach”. However, clinical research is capable of measuring many different facets of health via the numerous outcomes of the clinical encounter: from patient satisfaction, mood state, mental health, quality-of-life and positive outlook, to physical function, disability, pain intensity and recurrence. It is unlikely that this wide range of outcomes is assessed during the normal course of daily practice, and if patients are only asked how they feel, or “is their pain better?”, then perhaps this is a more reductionist approach to measure the impact of osteopathic treatment on an individual than the approach well designed clinical research employs.
This is where the numbers come into play, because changes that may follow osteopathic treatment in this wide range of patient domains are summarised by numbers in terms of statistical significance, confidence intervals, and importantly, the effect size (a measure of clinical relevance) of the treatment over and above other treatment approaches or natural history. It is also from these numbers that systematic reviewers, like Ernst and Canter, obtain the data from which systematic reviews and meta-analysis are prepared. Due to their position at the top of the evidence hierarchy, such systematic reviews have the potential to heavily influence healthcare policy and may therefore be considered one of the determinants of the future shape of the healthcare environment in which we all practice. Regardless of the methodological criticisms that can be aimed at the Ernst and Canter study, the overwhelming lesson is that we really must provide better numbers for systematic reviewers to work with.
The review by Ernst and Canter raises many issues, but perhaps the most important of these is that in order to answer such challenges, the profession will need to continue taking steps towards demonstrating the clinical effectiveness of osteopathic treatment using good quality clinical studies. Almost 10 years ago, Gibbons and Tehan6 wrote that the responsibility for the scientific credibility of osteopathic medicine rests solely with the osteopathic profession. They stated that:
“It is imperative that the osteopathic profession undertake research to validate clinical practice. A priority in research should be outcome studies to measure the impact of osteopathic treatment upon pain and disability.”
It is widely acknowledged that conducting studies to investigate clinical effectiveness is no simple undertaking. Organising the resources and expertise to undertake such studies usually requires the assembly of multidisciplinary teams. A decent clinical trial will require experienced and competent investigators who can prepare robust experimental designs, orchestrate grant writing to secure funding, gain ethical approvals, secure suitable clinical facilities, recruit and brief practitioners, and liaise with administrators, in addition to the actual recruitment, enrolment and ongoing management of patients. Then of course, there is data analysis, manuscript preparation and publication.
It's easy to reflect on Gibbons and Tehan's call and wonder what progress there has been towards satisfying the goal of having our own ‘osteopathic’ data to support the anecdotal success we collectively claim. One of the major difficulties is that there aren't enough experienced and available personnel within the profession to undertake high quality effectiveness studies. There have been very few major clinical outcome studies investigating osteopathy published over the last 10 years, and readers may skim through this journal wondering how many of the studies published in IJOM are relevant to clinical practice. It's easy to be nonchalant about the importance of research, and it's easy to dismiss as irrelevant the small-scale investigations into ‘how x technique changes y range of movement’, or the ‘number of consultations for x condition in y clinical practice’. In addition to the published results, a less obvious but important outcome from these studies is the hard won experience gained through wrangling with an ethics committee, or grappling with a funding proposal, or gaining more sophisticated data analysis skills through determination and strong espresso. These are among the skills that are needed to even draw near to the starting line for conducting a good quality effectiveness trial. Funding bodies won't grant money to researchers who lack a ‘track record’ – and there is only one way to develop a track record – we have to start with small steps.
So, for now the profession will struggle to respond to calls for effectiveness data, and for the interim it may be the apparently less important studies that will populate these pages. However, the profession is incrementally building expertise and experience in planning and conducting good science – exactly the skills needed to move into the clinical effectiveness arena.
References
- . A systematic review of systematic reviews of spinal manipulation. J R Soc Med. 2006;99:192–196
- . Systematic review of spinal manipulation: a balanced review of evidence?. J R Soc Med. 2006;99:277
- . Systematic review of spinal manipulation: flaws in the review. J R Soc Med. 2006;99:277–278
- . Systematic review of spinal manipulation: a biased report. J R Soc Med. 2006;99:278
- . Systematic review of spinal manipulation: including different techniques. J R Soc Med. 2006;99:277–279
- . Osteopathic medicine: validation of clinical practice by research. J Osteopath Educ Clin Res. 1997;7:10–18
PII: S1746-0689(06)00094-0
doi:10.1016/j.ijosm.2006.07.005
© 2006 Published by Elsevier Inc.
