Video-based assessment of practical operative skills for Undergraduate dental students

Abstract

Introduction: The aim of this study is to evaluate, within the scope of an experimental design, to what extent the assessment of two different settings of prepared cavities, based on video sequences, containing digital analysis tools of the prepCheck software, as well as to what extent they deviate from one another and are reliable.
Materials and Methods: For this prospective, single-centred, experimental study, 60 examination cavities related to a ceramic inlay preparation were assessed by four trainers in two different settings (A: video film versus B: video film plus an analogue model assessment) by using a standard checklist. The examined parameters contained: the 1. preparation / outer edges, 2. surface & smoothness / inner edges, 3. width & depth, 4. slide-in direction, 5. outer contact positioning and 6. overall grade on a Likert scale of 1 = ‘excellent’, 2 = ‘very good’, 3 = ‘good’, 4 = ‘satisfactory’ to 5 = ‘unsatisfactory’. An evaluation questionnaire with 33 items was additionally addressed to the concept of application of a digital-analytic software. The statistical analysis, using SAS 9.2 (SAS Institute Inc., Cary, USA, PROC MIXED) and R (Version 2.15, Package lme4) concerned the reliability, inter-rater correlation and significant factors at a p of 0.05.
Results: The assessment of the individual criteria and overall grade of the control group (A) were, on average, lower (i.e. better) than in the study group (B), yet with the exception of the ‘outer contact positioning’, without conclusive statistical significance. The reliability lay at an average of α=0.83 (A) and α=0.79 (B). The maximum reliability of the criteria ‘preparation edge’, ‘surface’, ‘width & depth’ as well as ‘overall grade’ were reasonable in the assessment mode, with α > 0.7. The inter Video-based Assessment 3 rater correlation was at an average of 0.43 < r < 0.74 higher in assessment mode A than B that comprised 0.35 < r < 0.60.
Conclusion: The current examination shows an average reliability in the assessment mode A that exceeds the requirements for practical examination (α ≥ 0.6) and also fulfils the general requirements for ‘high-stake’ examinations of α ≥ 0.8.