Assessing clinical competence in osteopathic education: Analysis of outcomes of different assessment strategies at the British School of Osteopathy

Abstract 

Background

Clinical competence is a complex concept, incorporating academic and operational competences and attitudinal meta-competences, which can be difficult to define and measure. Requirements for clinical competence in UK Osteopathy were broadened by the General Osteopathic Council in 1998 with changes in governance of Final Clinical Competence Assessments (FCCAs), prompting re-evaluation of assessments at the British School of Osteopathy (BSO).

Objectives

To analyse outcomes of two different methods of assessing clinical competence and evaluate their validity and congruence with professional standards.

Method

Retrospective statistical analysis of grades from FCCA and Clinic Tutor Reports (CTRs) for 6 cohorts of students graduating in 1998–2000 (n=240) and 2004–2006 (n=228).

Results

Mean scores for CTRs were higher than FCCAs in 1998–2000 with divergence increasing in 2004–2006 and fewer students failing CTRs. Correlation between FCCA and CTR grades was low in 1998 (r_s=0.34, p<0.01) and decreased to non-significance by 2000 (r_s=0.17, p>0.05). However, correlations increased steadily from 2004 (r_s=0.38, p<0.01) to 2006 (r_s=0.67, p<0.01). ‘Academic Knowledge’ was the only individual assessment item which was strongly correlated with Overall CTR grade and FCCA results in 1998–2000 but was not a separate assessment item in 2004–2006.

Conclusions

Weak correlations between FCCA and CTR grades in 1998 and stronger correlations in 2006 may demonstrate increasing convergence between the two assessments. However, health-care literature suggests that they measure different domains of competence, and congruent results are therefore not expected or desirable. Stronger correlations may indicate narrowing assessment focus and further research is necessary to clarify a broader range of assessment methods and criteria for assessing clinical competence in osteopathy.

Keywords: Education, Assessment, Clinical competence, Osteopathy, Osteopathic medicine