Advertisement

Reporting results in manual therapy clinical trials: A need for improvement

  • Author Footnotes
    1 RNC and GA contributed equally.
    Rodrigo Núñez-Cortés
    Correspondence
    Corresponding author. Department of Physical Therapy, Faculty of Medicine, University of Chile, Avenida Independencia 1027, Independencia, Santiago, Chile.
    Footnotes
    1 RNC and GA contributed equally.
    Affiliations
    Department of Physical Therapy, Faculty of Medicine, University of Chile, Santiago, Chile
    Search for articles by this author
  • Author Footnotes
    1 RNC and GA contributed equally.
    ,
    Author Footnotes
    2 Gerard Alvarez is a PhD student on Biomedical Research Methodology and Public Health in the Medical Department of the Universitat Autonoma de Barcelona, Barcelona, Spain.
    Gerard Alvarez
    Correspondence
    Corresponding author. Iberoamerican Cochrane Center C, Sant Antoni Maria Claret 167 Pavelló 18, Planta 0 08025 Barcelona, Catalunya, Spain.
    Footnotes
    1 RNC and GA contributed equally.
    2 Gerard Alvarez is a PhD student on Biomedical Research Methodology and Public Health in the Medical Department of the Universitat Autonoma de Barcelona, Barcelona, Spain.
    Affiliations
    Iberoamerican Cochrane Centre - Biomedical Research Institute Sant Pau, IIB Sant Pau, Barcelona, Spain

    Centre for Osteopathic Medicine - C.O.M.E. Collaboration, Spain National Center, Barcelona, Spain
    Search for articles by this author
  • Javier Pérez-Bracchiglione
    Affiliations
    Interdisciplinary Centre for Health Studies (CIESAL), Universidad de Valparaíso, Valparaíso, Chile
    Search for articles by this author
  • Rosa Cabanas-Valdés
    Affiliations
    Universitat Internacional de Catalunya, Physiotherapy Department, Faculty of Medicine and Health Sciences, Barcelona, Spain
    Search for articles by this author
  • Jordi Calvo-Sanz
    Affiliations
    Universitat Internacional de Catalunya, Physiotherapy Department, Faculty of Medicine and Health Sciences, Barcelona, Spain

    Departamento de Rehabilitación. Hospital - Centro de Prevención y Rehabilitación Asepeyo. St. Cugat del Vallés, Barcelona, Spain
    Search for articles by this author
  • Xavier Bonfill
    Affiliations
    Iberoamerican Cochrane Centre - Biomedical Research Institute Sant Pau, IIB Sant Pau, Barcelona, Spain

    CIBER Epidemiología y Salud Pública (CIBERESP), Spain
    Search for articles by this author
  • Gerard Urrutia
    Affiliations
    Iberoamerican Cochrane Centre - Biomedical Research Institute Sant Pau, IIB Sant Pau, Barcelona, Spain

    CIBER Epidemiología y Salud Pública (CIBERESP), Spain
    Search for articles by this author
  • Author Footnotes
    1 RNC and GA contributed equally.
    2 Gerard Alvarez is a PhD student on Biomedical Research Methodology and Public Health in the Medical Department of the Universitat Autonoma de Barcelona, Barcelona, Spain.

      Abstract

      Background

      The number of randomized clinical trials (RCTs) for manual therapy (MT) has increased exponentially in recent years but the quality of reporting is heterogeneous.

      Objective

      To assess the quality of the reporting of results in RCTs manual therapy, both in the text and in the graphs.

      Study design

      Methodological review.

      Methods

      We reviewed a random sample of 120 RCTs in MT published between 2000 and 2020 in indexed journals. We identified the primary outcome for each trial, and evaluated the completeness and correctness of reporting of results in the text and in the graphs.

      Results

      Forty per cent of the RCTs explicitly identified the primary outcome and 47.5% reported a sample size calculation. In 46.7% of the trials, the reporting of between groups comparisons was complete (including effect size and precision). Only 29.2% used the confidence interval as a measure of precision. Fifty-eight per cent of the trials reported significant differences in the results, and 30.8% reported a value of clinical relevance for at least one variable of the study. Forty-seven per cent reported the primary outcome graphically but only 19.6% of the graphs were self explanatory and 66.1% had problems of visual clarity.

      Conclusions

      Our findings suggest that the reporting of the results in MT trials is generally incomplete and graphics are often poor. These shortcomings could affect the interpretation of the results and their application in clinical practice. Improvements are needed in the reporting of results in order to advance clinical practice and research in manual therapy.

      Keywords

      To read this article in full you will need to make a payment

      References

        • French H.P.
        • Brennan A.
        • White B.
        • Cusack T.
        Manual therapy for osteoarthritis of the hip or knee - a systematic review.
        Man Ther. 2011; 16: 109-117
        • Alvarez G.
        • Solà I.
        • Sitjà-Rabert M.
        • Fort-Vanmeerhaeghe A.
        • Gich I.
        • Fernández C.
        • et al.
        A methodological review revealed that reporting of trials in manual therapy has not improved over time.
        J Clin Epidemiol. 2020; 121: 32-44https://doi.org/10.1016/j.jclinepi.2020.01.006
        • Shiwa S.R.
        • Moseley A.M.
        • Maher C.G.
        • Costa L.O.P.
        Language of publication has a small influence on the quality of reports of controlled trials of physiotherapy interventions.
        J Clin Epidemiol. 2013; 66: 78-84https://doi.org/10.1016/j.jclinepi.2012.08.004
        • Zavala-González J.
        • Pavez-Baeza F.
        • Gutiérrez-Espinoza H.
        • Olguín-Huerta C.
        The effectiveness of joint mobilization techniques for range of motion in adult patients with primary adhesive capsulitis of the shoulder: a systematic review and meta-analysis.
        Medwave. 2018; 18e7265
        • Minkalis A.L.
        • Vining R.D.
        • Long C.R.
        • Hawk C.
        • de Luca K.
        A systematic review of thrust manipulation combined with one conservative intervention for rotator cuff and related non-surgical shoulder conditions.
        J Can Chiropr Assoc. 2018; 62: 5-17
        • Carnes D.
        • Plunkett A.
        • Ellwood J.
        • Miles C.
        Manual therapy for unsettled, distressed and excessively crying infants: a systematic review and meta-analyses.
        BMJ Open. 2018; 8e019040
        • de Luca K.E.
        • Fang S.H.
        • Ong J.
        • Shin K.-S.
        • Woods S.
        • Tuchin P.J.
        The effectiveness and safety of manual therapy on pain and disability in older persons with chronic low back pain: a systematic review.
        J Manip Physiol Ther. 2017; 40: 527-534
        • Bizzarri P.
        • Buzzatti L.
        • Cattrysse E.
        • Scafoglieri A.
        Thoracic manual therapy is not more effective than placebo thoracic manual therapy in patients with shoulder dysfunctions: a systematic review with meta-analysis.
        Musculoskeletal Sci Pract. 2018; 33: 1-10
        • Laimi K.
        • Mäkilä A.
        • Bärlund E.
        • Katajapuu N.
        • Oksanen A.
        • Seikkula V.
        • et al.
        Effectiveness of myofascial release in treatment of chronic musculoskeletal pain: a systematic review.
        Clin Rehabil. 2018; 32: 440-450
        • Wu J.
        • Wang Y.
        • Wang Z.
        The effectiveness of massage and touch on behavioural and psychological symptoms of dementia: a quantitative systematic review and meta-analysis.
        J Adv Nurs. 2017; 73: 2283-2295
        • Smith C.A.
        • Levett K.M.
        • Collins C.T.
        • Dahlen H.G.
        • Ee C.C.
        • Suganuma M.
        Massage, reflexology and other manual methods for pain management in labour.
        Cochrane Database Syst Rev. 2018; 3: CD009290
        • Furlan A.D.
        • Giraldo M.
        • Baskwill A.
        • Irvin E.
        • Imamura M.
        Massage for low-back pain.
        Cochrane Database Syst Rev. 2015; https://doi.org/10.1002/14651858.cd001929.pub3
        • Westad K.
        • Tjoestolvsen F.
        • Hebron C.
        The effectiveness of Mulligan's mobilisation with movement (MWM) on peripheral joints in musculoskeletal (MSK) conditions: a systematic review.
        Musculoskeletal Sci Pract. 2018; https://doi.org/10.1016/j.msksp.2018.12.001
        • Lee N.-W.
        • Kim G.-H.
        • Heo I.
        • Kim K.-W.
        • Ha I.-H.
        • Lee J.-H.
        • et al.
        Chuna (or tuina) manual therapy for musculoskeletal disorders: a systematic review and meta-analysis of randomized controlled trials.
        Evid Based Compl Alternat Med. 2017; (2017)8218139
        • Xu Q.
        • Chen B.
        • Wang Y.
        • Wang X.
        • Han D.
        • Ding D.
        • et al.
        The effectiveness of manual therapy for relieving pain, stiffness, and dysfunction in knee osteoarthritis: a systematic review and meta-analysis.
        Pain Physician. 2017; 20: 229-243
        • Mischke J.J.
        • Jayaseelan D.J.
        • Sault J.D.
        • Emerson Kavchak A.J.
        The symptomatic and functional effects of manual physical therapy on plantar heel pain: a systematic review.
        J Man Manip Ther. 2017; 25: 3-10
        • Krul M.
        • van der Wouden J.C.
        • Kruithof E.J.
        • van Suijlekom-Smit L.W.
        • Koes B.W.
        Manipulative interventions for reducing pulled elbow in young children.
        Cochrane Database Syst Rev. 2017; 7: CD007759
        • Gross A.
        • Langevin P.
        • Burnie S.J.
        • Bédard-Brochu M.-S.
        • Empey B.
        • Dugas E.
        • et al.
        Manipulation and mobilisation for neck pain contrasted against an inactive control or another active treatment.
        Cochrane Database Syst Rev. 2015; https://doi.org/10.1002/14651858.cd004249.pub4
        • Silva T.M.
        • Costa L.C.M.
        • Costa L.O.P.
        Evidence-Based Practice: a survey regarding behavior, knowledge, skills, resources, opinions and perceived barriers of Brazilian physical therapists from São Paulo state.
        Braz J Phys Ther. 2015; 19: 294-303
        • da Silva T.M.
        • Costa L. da CM.
        • Garcia A.N.
        • Costa L.O.P.
        What do physical therapists think about evidence-based practice? A systematic review.
        Man Ther. 2015; 20: 388-401
        • Straulino S.
        Reconstruction of Galileo Galilei's experiment: the inclined plane.
        Phys Educ. 2008; 43: 316-321https://doi.org/10.1088/0031-9120/43/3/012
        • Altman D.G.
        Better reporting of randomised controlled trials: the CONSORT statement.
        BMJ. 1996; 313: 570-571
        • Caulley L.
        • Catalá-López F.
        • Whelan J.
        • Khoury M.
        • Ferraro J.
        • Cheng W.
        • et al.
        Reporting guidelines of health research studies are frequently used inappropriately.
        J Clin Epidemiol. 2020; 122: 87-94
        • Moseley A.M.
        • Elkins M.R.
        • Janer-Duncan L.
        • Hush J.M.
        The quality of reports of randomized controlled trials varies between subdisciplines of physiotherapy.
        Physiother Can. 2014; 66: 36-43
        • Moseley A.M.
        • Herbert R.D.
        • Maher C.G.
        • Sherrington C.
        • Elkins M.R.
        Reported quality of randomized controlled trials of physiotherapy interventions has improved over time.
        J Clin Epidemiol. 2011; 64: 594-601
        • Riley S.P.
        • Swanson B.
        • Brismée J.-M.
        • Sawyer S.F.
        A systematic review of orthopaedic manual therapy randomized clinical trials quality.
        J Man Manip Ther. 2016; 24: 241-252
        • Puhan M.A.
        • ter Riet G.
        • Eichler K.
        • Steurer J.
        • Bachmann L.M.
        More medical journals should inform their contributors about three key principles of graph construction.
        J Clin Epidemiol. 2006; 59: 1017-1022
        • Pocock S.J.
        • Travison T.G.
        • Wruck L.M.
        Figures in clinical trial reports: current practice & scope for improvement.
        Trials. 2007; 8: 36
        • Altman D.G.
        • Schulz K.F.
        • Moher D.
        • Egger M.
        • Davidoff F.
        • Elbourne D.
        • et al.
        The revised CONSORT statement for reporting randomized trials: explanation and elaboration.
        Ann Intern Med. 2001; 134: 663-694
        • Pillastrini P.
        • Vanti C.
        • Curti S.
        • Mattioli S.
        • Ferrari S.
        • Violante F.S.
        • et al.
        Using PubMed search strings for efficient retrieval of manual therapy research literature.
        J Manip Physiol Ther. 2015; 38: 159-166
        • Butcher N.J.
        • Monsour A.
        • Mew E.J.
        • Szatmari P.
        • Pierro A.
        • Kelly L.E.
        • et al.
        Improving outcome reporting in clinical trial reports and protocols: study protocol for the Instrument for reporting Planned Endpoints in Clinical Trials (InsPECT).
        Trials. 2019; 20: 161
        • Butcher N.J.
        • Mew E.J.
        • Saeed L.
        • Monsour A.
        • Chee-A-Tow A.
        • Chan A.-W.
        • et al.
        Guidance for reporting outcomes in clinical trials: scoping review protocol.
        BMJ Open. 2019; 9e023001
        • Chen J.C.
        • Cooper R.J.
        • McMullen M.E.
        • Schriger D.L.
        Graph quality in top medical journals.
        Ann Emerg Med. 2017; 69 (61.e5): 453
        • Landis J.R.
        • Koch G.G.
        The measurement of observer agreement for categorical data.
        Biometrics. 1977; 33: 159-174
        • Bhaloo Z.
        • Adams D.
        • Liu Y.
        • Hansraj N.
        • Hartling L.
        • Terwee C.B.
        • et al.
        Primary Outcomes Reporting in Trials (PORTal): a systematic review of inadequate reporting in pediatric randomized controlled trials.
        J Clin Epidemiol. 2017; 81: 33-41
        • Sinha I.P.
        • Altman D.G.
        • Beresford M.W.
        • Boers M.
        • Clarke M.
        • Craig J.
        • et al.
        Standard 5: selection, measurement, and reporting of outcomes in clinical trials in children.
        Pediatrics. 2012; 129: S146-S152
        • Mayo-Wilson E.
        • Fusco N.
        • Li T.
        • Hong H.
        • Canner J.K.
        • Dickersin K.
        • et al.
        Multiple outcomes and analyses in clinical trials create challenges for interpretation and research synthesis.
        J Clin Epidemiol. 2017; 86: 39-50
        • Mantziari S.
        • Demartines N.
        Poor outcome reporting in medical research; building practice on spoilt grounds.
        Ann Transl Med. 2017; 5: S15
        • Dwan K.
        • Gamble C.
        • Williamson P.R.
        • Kirkham J.J.
        Reporting Bias Group. Systematic review of the empirical evidence of study publication bias and outcome reporting bias - an updated review.
        PloS One. 2013; 8e66844
        • Hall N.J.
        • Kapadia M.Z.
        • Eaton S.
        • Chan W.W.Y.
        • Nickel C.
        • Pierro A.
        • et al.
        Outcome reporting in randomised controlled trials and meta-analyses of appendicitis treatments in children: a systematic review.
        Trials. 2015; 16: 275
        • Saldanha I.J.
        • Dickersin K.
        • Wang X.
        • Li T.
        Outcomes in Cochrane systematic reviews addressing four common eye conditions: an evaluation of completeness and comparability.
        PloS One. 2014; 9e109400
        • Chan A.-W.
        • Altman D.G.
        Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors.
        BMJ. 2005; 330: 753
        • Froud R.
        • Underwood M.
        • Eldridge S.
        Improving the reporting and interpretation of clinical trial outcomes.
        Br J Gen Pract. 2012; 62: e729-e731https://doi.org/10.3399/bjgp12x657008
        • Heneghan C.
        • Goldacre B.
        • Mahtani K.R.
        Why clinical trial outcomes fail to translate into benefits for patients.
        Trials. 2017; 18: 122
        • Vetter T.R.
        • Mascha E.J.
        Defining the primary outcomes and justifying secondary outcomes of a study: usually, the fewer, the better.
        Anesth Analg. 2017; 125: 678-681
        • Abdul Latif L.
        • Daud Amadera J.E.
        • Pimentel D.
        • Pimentel T.
        • Fregni F.
        Sample size calculation in physical medicine and rehabilitation: a systematic review of reporting, characteristics, and results in randomized controlled trials.
        Arch Phys Med Rehabil. 2011; 92: 306-315
        • Castellini G.
        • Gianola S.
        • Bonovas S.
        • Moja L.
        Improving power and sample size calculation in rehabilitation trial reports: a methodological assessment.
        Arch Phys Med Rehabil. 2016; 97: 1195-1201
        • Koes B.W.
        How to evaluate manual therapy: value and pitfalls of randomized clinical trials.
        Man Ther. 2004; 9: 183-184
        • Rubinstein S.M.
        • van Eekelen R.
        • Oosterhuis T.
        • de Boer M.R.
        • Ostelo R.W.J.G.
        • van Tulder M.W.
        The risk of bias and sample size of trials of spinal manipulative therapy for low back and neck pain: analysis and recommendations.
        J Manip Physiol Ther. 2014; 37: 523-541
        • Page P.
        Beyond statistical significance: clinical interpretation of rehabilitation research literature.
        Int J Sports Phys Ther. 2014; 9: 726-736
        • Armijo-Olivo S.
        The importance of determining the clinical significance of research results in physical therapy clinical research.
        Braz J Phys Ther. 2018; 22: 175-176
        • Armijo-Olivo S.
        • Warren S.
        • Fuentes J.
        • Magee D.J.
        Clinical relevance vs. statistical significance: using neck outcomes in patients with temporomandibular disorders as an example.
        Man Ther. 2011; 16: 563-572
        • Rossettini G.
        • Carlino E.
        • Testa M.
        Clinical relevance of contextual factors as triggers of placebo and nocebo effects in musculoskeletal pain.
        BMC Muscoskel Disord. 2018; 19https://doi.org/10.1186/s12891-018-1943-8
        • Bialosky J.E.
        • Beneciuk J.M.
        • Bishop M.D.
        • Coronado R.A.
        • Penza C.W.
        • Simon C.B.
        • et al.
        Unraveling the mechanisms of manual therapy: modeling an approach.
        J Orthop Sports Phys Ther. 2018; 48: 8-18
        • Bialosky J.E.
        • Bishop M.D.
        • Penza C.W.
        Placebo mechanisms of manual therapy: a sheep in wolf's clothing?.
        J Orthop Sports Phys Ther. 2017; 47: 301-304
        • Kamper S.J.
        Interpreting outcomes 1—change and difference: linking evidence to practice.
        J Orthop Sports Phys Ther. 2019; 49: 357-358
        • Goodman S.
        A dirty dozen: twelve P-value misconceptions.
        Semin Hematol. 2008; 45: 135-140https://doi.org/10.1053/j.seminhematol.2008.04.003
        • Oliveri R.S.
        • Gluud C.
        • Wille-Jørgensen P.A.
        Hospital doctors' self-rated skills in and use of evidence-based medicine - a questionnaire survey.
        J Eval Clin Pract. 2004; 10: 219-226
        • Greenland S.
        • Senn S.J.
        • Rothman K.J.
        • Carlin J.B.
        • Poole C.
        • Goodman S.N.
        • et al.
        Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations.
        Eur J Epidemiol. 2016; 31: 337-350
        • Bassler D.
        • Briel M.
        • Montori V.M.
        • Lane M.
        • Glasziou P.
        • Zhou Q.
        • et al.
        Stopping randomized trials early for benefit and estimation of treatment effects: systematic review and meta-regression analysis.
        J Am Med Assoc. 2010; 303: 1180-1187
        • Bleakley C.
        • Reijgers J.
        • Smoliga J.M.
        Many high-quality randomized controlled trials in sports physical therapy are making false-positive claims of treatment effect: a systematic survey.
        J Orthop Sports Phys Ther. 2020; 50: 104-109https://doi.org/10.2519/jospt.2020.9264
        • Fethney J.
        Statistical and clinical significance, and how to use confidence intervals to help interpret both.
        Aust Crit Care. 2010; 23: 93-97
        • Kamper S.J.
        Confidence intervals: linking evidence to practice.
        J Orthop Sports Phys Ther. 2019; 49: 763-764
        • Armijo-Olivo S.
        • Warren S.
        • Magee D.
        Intention to treat analysis, compliance, drop-outs and how to deal with missing data in clinical research: a review.
        Phys Ther Rev. 2009; 14: 36-49
        • von Allmen R.S.
        • Weiss S.
        • Tevaearai H.T.
        • Kuemmerli C.
        • Tinner C.
        • Carrel T.P.
        • et al.
        Completeness of follow-up determines validity of study findings: results of a prospective repeated measures cohort study.
        PloS One. 2015; 10e0140817
        • Moher D.
        • Hopewell S.
        • Schulz K.F.
        • Montori V.
        • Gøtzsche P.C.
        • Devereaux P.J.
        • et al.
        CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials.
        BMJ. 2010; 340 (10 1136/bmj.c869): c869
        • Akl E.A.
        • Briel M.
        • You J.J.
        • Sun X.
        • Johnston B.C.
        • Busse J.W.
        • et al.
        Potential impact on estimated treatment effects of information lost to follow-up in randomised controlled trials (LOST-IT): systematic review.
        BMJ. 2012; 344e2809
        • Schriger D.L.
        • Sinha R.
        • Schroter S.
        • Liu P.Y.
        • Altman D.G.
        From submission to publication: a retrospective review of the tables and figures in a cohort of randomized controlled trials submitted to the British Medical Journal.
        Ann Emerg Med. 2006; 48 (750–6, 756.e1–21.)
        • Schriger D.L.
        • Cooper R.J.
        Achieving graphical excellence: suggestions and methods for creating high-quality visual displays of experimental data.
        Ann Emerg Med. 2001; 37: 75-87
        • Pocock S.J.
        • Travison T.G.
        • Wruck L.M.
        How to interpret figures in reports of clinical trials.
        BMJ. 2008; 336: 1166-1169
        • Cleveland W.S.
        The elements of graphing data.
        1992
        • Blanco D.
        • Altman D.
        • Moher D.
        • Boutron I.
        • Kirkham J.J.
        • Cobo E.
        Scoping review on interventions to improve adherence to reporting guidelines in health research.
        BMJ Open. 2019; 9e026589
        • Hopewell S.
        • Boutron I.
        • Altman D.G.
        • Barbour G.
        • Moher D.
        • Montori V.
        • et al.
        Impact of a web-based tool (WebCONSORT) to improve the reporting of randomised trials: results of a randomised controlled trial.
        BMC Med. 2016; 14: 199
        • Barnes C.
        • Boutron I.
        • Giraudeau B.
        • Porcher R.
        • Altman D.G.
        • Ravaud P.
        Impact of an online writing aid tool for writing a randomized trial report: the COBWEB (Consort-based WEB tool) randomized controlled trial.
        BMC Med. 2015; 13: 221