Small patient samples can affect the accuracy of radiology research

Diagnostic imaging research is replete with studies with smaller sample sizes, which could lead to the exaggeration of larger and more favorable effects than might be found in studies of patient populations. more important. Researchers led by Dr Lucy Lu of Flinders University in Bedford Park wrote that evidence for this event – also known as the small study effect – can be found in their analysis of nearly 670 studies. and more than 80,000 patients.

“These results have important implications for the conduct and interpretation of meta-analyses in the diagnostic imaging literature, as they suggest that the estimates of diagnostic accuracy presented by many meta-analyses may be gross overestimates” , wrote Lu et al.

The idea behind small study effects is that studies with smaller sample sizes tend to report larger and more favorable effect estimates than studies with larger sample sizes. The most common reasons for this to happen are bias, heterogeneity, and pure coincidence.

Small studies are more prone to publication bias, in which manuscripts with statistically significant or favorable results are more likely to be published. Outcome reporting bias, where selective reporting of only the most favorable outcomes occurs, is another contributor to the effects of small studies.

The effects of small studies caused by bias can impact meta-analyses by increasing the estimated pooled effect sizes. Detecting such bias by retrieving unpublished data is usually not possible, so statistical tools such as funnel plots, Egger’s test, and Deek’s test are used to detect these effects.

In diagnostic imaging, the effects of small studies are not well understood. The researchers pointed out that “most” diagnostic accuracy studies do not have a predefined study hypothesis, while most randomized clinical trials of interventions test a predefined hypothesis.

Lu’s team wanted to uncover the presence and magnitude of the effects of small studies in diagnostic imaging accuracy meta-analyses. They reviewed data from 31 meta-analyses published between 2010 and 2019. The team included meta-analyses with 10 or more studies on the diagnostic accuracy of medical imaging, evaluating a single imaging modality. A total of 80,206 patients were included in these meta-analyses.

The researchers also used the natural logarithm of the diagnostic odds ratio (ln[DOR]) to detect publication bias. The diagnostic odds ratio shows the effectiveness of a diagnostic test. From this, they created a composite funnel plot by plotting the effect size of the study against the precision of all the primary studies.

The most studied imaging modalities were MRI (11 meta-analyses), ultrasound (8 meta-analyses), and computed tomography (5 meta-analyses), with the other modalities studied in two or fewer meta-analyses .

By studying a composite funnel plot and using regression analysis, the researchers found that a “relative lack” of studies had higher study effects, with similar trends seen in individual imaging modalities. . They also found a regression coefficient of ln(DOR) on study effects of ln(DOR) of 2.19 (p

The team also found an inverse association between effect size estimation and accuracy independent of imaging modality.

In other words, the larger the study, the more precise its effects. The smaller the study, the more inflated its effects in diagnostic imaging.

“Since studies with lower precision generally represent those with smaller sample sizes, these results are consistent with small study effects,” Lu and colleagues wrote. “The observation was consistent across all individual modalities.”

The team also found that of 26 meta-analyses formally assessing publication bias using funnel plots and statistical tests, 21 found no evidence of such bias.

The authors called for further research to uncover the different factors contributing to the effects of small studies. They wrote that these effects have implications for costs, imaging utilization, and patient outcomes due to suboptimal imaging test choice or unnecessary exposure to ionizing radiation.

“To alleviate some of these issues, we suggest that readers and authors be aware that commonly used tests for funnel plot asymmetry are often underpowered in diagnostic imaging meta-analyses and therefore may underestimating or failing to detect the presence of an underlying bias,” they added.

Copyright © 2022

Comments are closed.