TY - JOUR
T1 - RoB-SPEO
T2 - A tool for assessing risk of bias in studies estimating the prevalence of exposure to occupational risk factors from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury
AU - Pega, Frank
AU - Norris, Susan L
AU - Backes, Claudine
AU - Bero, Lisa A
AU - Descatha, Alexis
AU - Gagliardi, Diana
AU - Godderis, Lode
AU - Loney, Tom
AU - Modenese, Alberto
AU - Morgan, Rebecca L
AU - Pachito, Daniela
AU - Paulo, Marilia B S
AU - Scheepers, Paul T J
AU - Schlünssen, Vivi
AU - Sgargi, Daria
AU - Silbergeld, Ellen K
AU - Sørensen, Kathrine
AU - Sutton, Patrice
AU - Tenkate, Thomas
AU - Torreão Corrêa da Silva, Denise
AU - Ujita, Yuka
AU - van Deventer, Emilie
AU - Woodruff, Tracey J
AU - Mandrioli, Daniele
N1 - Copyright © 2019 World Health Organization, International Labour Organization. Published by Elsevier Ltd.. All rights reserved.
PY - 2020/2
Y1 - 2020/2
N2 - BACKGROUND: The World Health Organization (WHO) and the International Labour Organization (ILO) are developing joint estimates of the work-related burden of disease and injury (WHO/ILO Joint Estimates). For this, systematic reviews of studies estimating the prevalence of exposure to selected occupational risk factors will be conducted to provide input data for estimations of the number of exposed workers. A critical part of systematic review methods is to assess risk of bias (RoB) of individual studies. In this article, we present and describe the development of such a tool, called the Risk of Bias in Studies estimating Prevalence of Exposure to Occupational risk factors (RoB-SPEO) tool; report results from RoB-SPEO's pilot testing; note RoB-SPEO's limitations; and suggest how the tool might be tested and developed further.METHODS: Selected existing RoB tools used in environmental and occupational health systematic reviews were reviewed and analysed. From existing tools, we identified domains for the new tool and, if necessary, added new domains. For each domain, we then identified and integrated components from the existing tools (i.e. instructions, domains, guiding questions, considerations, ratings and rating criteria), and, if necessary, we developed new components. Finally, we elicited feedback from other systematic review methodologists and exposure scientists and agreed upon RoB-SPEO. Nine experts pilot tested RoB-SPEO, and we calculated a raw measure of inter-rater agreement (Pi) for each of its domain, rating Pi < 0.4 as poor, 0.4 ≤ Pi ≥ 0.8 as substantial and Pi > 0.80 as almost perfect agreement.RESULTS: Our review found no standard tool for assessing RoB in prevalence studies of exposure to occupational risk factors. We identified six existing tools for environmental and occupational health systematic reviews and found that their components for assessing RoB differ considerably. With the new RoB-SPEO tool, assessors judge RoB for each of eight domains: (1) bias in selection of participants into the study; (2) bias due to a lack of blinding of study personnel; (3) bias due to exposure misclassification; (4) bias due to incomplete exposure data; (5) bias due to conflict of interest; (6) bias due to selective reporting of exposures; (7) bias due to difference in numerator and denominator; and (8) other bias. The RoB-SPEO's ratings are low, probably low, probably high, high or no information. Pilot testing of the RoB-SPEO tool found substantial inter-rater agreement for six domains (range of Pi for these domains: 0.51-0.80), but poor agreement for two domains (i.e. Pi of 0.31 and 0.33 for biases due to incomplete exposure data and in selection of participants into the study, respectively). Limitations of RoB-SPEO include that it has not yet been fully performance-tested.CONCLUSIONS: We developed the RoB-SPEO tool for assessing RoB in prevalence studies of exposure to occupational risk factors. The tool will be applied and its performance tested in the ongoing systematic reviews for the WHO/ILO Joint Estimates.
AB - BACKGROUND: The World Health Organization (WHO) and the International Labour Organization (ILO) are developing joint estimates of the work-related burden of disease and injury (WHO/ILO Joint Estimates). For this, systematic reviews of studies estimating the prevalence of exposure to selected occupational risk factors will be conducted to provide input data for estimations of the number of exposed workers. A critical part of systematic review methods is to assess risk of bias (RoB) of individual studies. In this article, we present and describe the development of such a tool, called the Risk of Bias in Studies estimating Prevalence of Exposure to Occupational risk factors (RoB-SPEO) tool; report results from RoB-SPEO's pilot testing; note RoB-SPEO's limitations; and suggest how the tool might be tested and developed further.METHODS: Selected existing RoB tools used in environmental and occupational health systematic reviews were reviewed and analysed. From existing tools, we identified domains for the new tool and, if necessary, added new domains. For each domain, we then identified and integrated components from the existing tools (i.e. instructions, domains, guiding questions, considerations, ratings and rating criteria), and, if necessary, we developed new components. Finally, we elicited feedback from other systematic review methodologists and exposure scientists and agreed upon RoB-SPEO. Nine experts pilot tested RoB-SPEO, and we calculated a raw measure of inter-rater agreement (Pi) for each of its domain, rating Pi < 0.4 as poor, 0.4 ≤ Pi ≥ 0.8 as substantial and Pi > 0.80 as almost perfect agreement.RESULTS: Our review found no standard tool for assessing RoB in prevalence studies of exposure to occupational risk factors. We identified six existing tools for environmental and occupational health systematic reviews and found that their components for assessing RoB differ considerably. With the new RoB-SPEO tool, assessors judge RoB for each of eight domains: (1) bias in selection of participants into the study; (2) bias due to a lack of blinding of study personnel; (3) bias due to exposure misclassification; (4) bias due to incomplete exposure data; (5) bias due to conflict of interest; (6) bias due to selective reporting of exposures; (7) bias due to difference in numerator and denominator; and (8) other bias. The RoB-SPEO's ratings are low, probably low, probably high, high or no information. Pilot testing of the RoB-SPEO tool found substantial inter-rater agreement for six domains (range of Pi for these domains: 0.51-0.80), but poor agreement for two domains (i.e. Pi of 0.31 and 0.33 for biases due to incomplete exposure data and in selection of participants into the study, respectively). Limitations of RoB-SPEO include that it has not yet been fully performance-tested.CONCLUSIONS: We developed the RoB-SPEO tool for assessing RoB in prevalence studies of exposure to occupational risk factors. The tool will be applied and its performance tested in the ongoing systematic reviews for the WHO/ILO Joint Estimates.
KW - Bias
KW - Humans
KW - Occupational Diseases
KW - Occupational Exposure
KW - Prevalence
KW - Risk Factors
KW - World Health Organization
KW - Wounds and Injuries/epidemiology
U2 - 10.1016/j.envint.2019.105039
DO - 10.1016/j.envint.2019.105039
M3 - Article
C2 - 31864023
SN - 0160-4120
VL - 135
SP - 105039
JO - Environment international
JF - Environment international
ER -