Usability and preference of electronic vs. paper and pencil OSCE checklists by examiners and influence of checklist type on missed ratings in the Swiss Federal Licensing Exam.

Wagner, Felicitas L; Feller, Sabine; Schmitz, Felix M; Zimmermann, Philippe G; Krings, Rabea; Guttormsen, Sissel; Huwendiek, Sören (2022). Usability and preference of electronic vs. paper and pencil OSCE checklists by examiners and influence of checklist type on missed ratings in the Swiss Federal Licensing Exam. GMS Journal for Medical Education, 39(2), Doc24. German Medical Science 10.3205/zma001545

[img]
Preview
Text
zma001545.pdf - Published Version
Available under License Creative Commons: Attribution (CC-BY).

Download (3MB) | Preview

Background

Only a few studies with small sample sizes have compared electronic Objective Structured Clinical Examination (OSCE) rating checklists with traditional paper-based OSCE rating checklists. In this study, the examiner-perceived usability and preference for type of OSCE checklist (electronic vs. paper based) were compared, and the influence of OSCE checklist type on missed ratings was determined, for the Swiss Federal Licensing Examination in clinical skills for human medicine.

Methods

All examiners in the Swiss Federal Licensing Examination in clinical skills for human medicine were invited over two subsequent years to evaluate the OSCE checklist type they had worked with during the examination. This was based on a questionnaire with 14 closed questions (i.e., demographic, checklist-type experience, perceived usability, checklist type preference). Furthermore, the numbers of missed ratings for the paper-based checklist were recorded.

Results

The data from these examiners (n=377) with experience of both OSCE checklist types were analyzed. The electronic OSCE checklist was rated significantly higher on all usability aspects (i.e., ease of use, candidate rating and error correction, clarity, distraction using the checklist, overall satisfaction), except for the speed of registering comments (no significant difference). The majority of the examiners in both years (2014: 54.5%, n=60, 2015: 89.8%, n=230) reported preference for working with the electronic OSCE checklist in the future. Missed ratings were seen for 14.2% of the paper-based OSCE checklists, which were prevented with the electronic OSCE checklists.

Conclusions

Electronic OSCE checklists were rated significantly more user-friendly and were preferred over paper-based OSCE checklists by a broad national sample of examiners, supporting previous results from faculty-level examinations. Furthermore, missed ratings were prevented with the electronic OSCE checklists. Overall, the use of electronic OSCE checklists is therefore advisable.

Item Type:

Journal Article (Original Article)

Division/Institute:

04 Faculty of Medicine > Medical Education > Institute for Medical Education
04 Faculty of Medicine > Medical Education > Institute for Medical Education > Assessment and Evaluation Unit (AAE)

UniBE Contributor:

Wagner, Felicitas Lony; Feller, Sabine; Schmitz, Felix Michael; Zimmermann, Philippe; Krings, Rabea; Guttormsen, Sissel and Huwendiek, Sören

Subjects:

600 Technology > 610 Medicine & health

ISSN:

2366-5017

Publisher:

German Medical Science

Language:

Multilingual

Submitter:

Pubmed Import

Date Deposited:

14 Jun 2022 12:02

Last Modified:

19 Jun 2022 01:58

Publisher DOI:

10.3205/zma001545

PubMed ID:

35692359

Additional Information:

Doc24

Uncontrolled Keywords:

OSCE checklists electronic evaluation national usability

BORIS DOI:

10.48350/170633

URI:

https://boris.unibe.ch/id/eprint/170633

Actions (login required)

Edit item Edit item
Provide Feedback