Do clinicians understand the size of treatment effects? A randomized survey across 8 countries.

Johnston, Bradley C; Alonso-Coello, Pablo; Friedrich, Jan O; Mustafa, Reem A; Tikkinen, Kari A O; Neumann, Ignacio; Vandvik, Per O; Akl, Elie A; Da Costa, Bruno R; Adhikari, Neill K; Dalmau, Gemma Mas; Kosunen, Elise; Mustonen, Jukka; Crawford, Mark W; Thabane, Lehana; Guyatt, Gordon H (2016). Do clinicians understand the size of treatment effects? A randomized survey across 8 countries. CMAJ, 188(1), pp. 25-32. Canadian Medical Association 10.1503/cmaj.150430

[img] Text
Johnston CMAJ 2016.pdf - Published Version
Restricted to registered users only
Available under License Publisher holds Copyright.

Download (160kB) | Request a copy

BACKGROUND Meta-analyses of continuous outcomes typically provide enough information for decision-makers to evaluate the extent to which chance can explain apparent differences between interventions. The interpretation of the magnitude of these differences - from trivial to large - can, however, be challenging. We investigated clinicians' understanding and perceptions of usefulness of 6 statistical formats for presenting continuous outcomes from meta-analyses (standardized mean difference, minimal important difference units, mean difference in natural units, ratio of means, relative risk and risk difference). METHODS We invited 610 staff and trainees in internal medicine and family medicine programs in 8 countries to participate. Paper-based, self-administered questionnaires presented summary estimates of hypothetical interventions versus placebo for chronic pain. The estimates showed either a small or a large effect for each of the 6 statistical formats for presenting continuous outcomes. Questions addressed participants' understanding of the magnitude of treatment effects and their perception of the usefulness of the presentation format. We randomly assigned participants 1 of 4 versions of the questionnaire, each with a different effect size (large or small) and presentation order for the 6 formats (1 to 6, or 6 to 1). RESULTS Overall, 531 (87.0%) of the clinicians responded. Respondents best understood risk difference, followed by relative risk and ratio of means. Similarly, they perceived the dichotomous presentation of continuous outcomes (relative risk and risk difference) to be most useful. Presenting results as a standardized mean difference, the longest standing and most widely used approach, was poorly understood and perceived as least useful. INTERPRETATION None of the presentation formats were well understood or perceived as extremely useful. Clinicians best understood the dichotomous presentations of continuous outcomes and perceived them to be the most useful. Further initiatives to help clinicians better grasp the magnitude of the treatment effect are needed.

Item Type:

Journal Article (Original Article)

Division/Institute:

04 Faculty of Medicine > Medical Education > Institute of General Practice and Primary Care (BIHAM)

UniBE Contributor:

Da Costa, Bruno

Subjects:

600 Technology > 610 Medicine & health
300 Social sciences, sociology & anthropology > 360 Social problems & social services

ISSN:

0820-3946

Publisher:

Canadian Medical Association

Language:

English

Submitter:

Doris Kopp Heim

Date Deposited:

12 Jan 2016 14:46

Last Modified:

06 Jul 2017 14:03

Publisher DOI:

10.1503/cmaj.150430

PubMed ID:

26504102

BORIS DOI:

10.7892/boris.74831

URI:

https://boris.unibe.ch/id/eprint/74831

Actions (login required)

Edit item Edit item
Provide Feedback