Expert panels as a reference standard in orthodontic research: An assessment of published methods and reporting

Lempesi, E; Toulia, E; Pandis, Nikolaos (2017). Expert panels as a reference standard in orthodontic research: An assessment of published methods and reporting. American journal of orthodontics and dentofacial orthopedics, 151(4), pp. 656-668. Elsevier 10.1016/j.ajodo.2016.09.020

[img] Text
PIIS0889540616308848.pdf - Published Version
Restricted to registered users only
Available under License Publisher holds Copyright.

Download (488kB) | Request a copy

INTRODUCTION: The aim of this study was to investigate the expert panel methodology applied in orthodontics and its reporting quality. Additionally, the relationship between the reporting quality and a range of variables was explored. METHODS: PubMed was searched for orthodontic studies in which the final diagnosis or assessment was made by 2 or more experts published up to March 16, 2015. Reporting quality assessment was conducted using an established modified checklist. The relationship between potential predictors and the total score was assessed using univariable linear regression. RESULTS: We identified 237 studies with a mean score of 9.97 (SD, 1.12) out of a maximum of 15. Critical information about panel methodology was missing in all studies. The panel composition differed substantially across studies, ranging from 2 to 646 panel members, with large variations in the expertise represented. Only 17 studies (7.2%) reported sample size calculations to justify the panel size. Panel members were partly blinded in 65 (27.4%) studies. Most studies failed to report which statistic was used to compute intrarater (65.8%) and interrater (66.2%) agreements. Journal type (nonorthodontic: β, 0.23; 95% CI, -0.07 to 0.54 compared with orthodontic), publication year (β, 0; 95% CI, -0.02 to 0.02 for each additional year), number of authors (1-3: β, 0.30; 95% CI, -0.13 to 0.74 compared with at least 6; 4-5: β, 0.18; 95% CI, -0.29 to 0.33 compared with at least 6), and number of centers involved (single: β, 0.20; 95% CI, -0.14 to 0.54 compared with multicenter) were not significant predictors of improved reporting. Studies published in Asia and Australia had significantly lower scores compared with those published in Europe (β, -0.54; 95% CI, -0.92 to -0.17). CONCLUSIONS: Formal guidelines on methodology and reporting of studies involving expert panels are required.

Item Type:

Journal Article (Original Article)


04 Faculty of Medicine > School of Dental Medicine > Department of Orthodontics

UniBE Contributor:

Pandis, Nikolaos


600 Technology > 610 Medicine & health








Renate Imhof-Etter

Date Deposited:

27 Mar 2018 11:08

Last Modified:

01 May 2018 17:01

Publisher DOI:


PubMed ID:





Actions (login required)

Edit item Edit item
Provide Feedback