Paper mills in evidence synthesis: reported involvement among retracted systematic reviews and meta-analyses.
Begagić Emir E, Skenderi Faruk F, Vranić Semir S
To investigate reported paper mill involvement among retracted systematic reviews and meta-analyses (SR/MAs), and to compare retracted SR/MAs with and without reported paper mill involvement with respect to temporal patterns, disciplinary profile, country recorded in the database, and retraction characteristics. Cross-sectional study based on the Retraction Watch database. Records were screened to identify retracted SR/MAs. Reported paper mill involvement was assigned only when explicitly stated in the Retraction Watch record or associated retraction information; fake peer review was coded separately. Group comparisons used the Pearson chi-square and Mann-Whitney U tests. Associations with reported paper mill involvement were estimated using univariable Poisson regression with robust variance and reported as prevalence ratios (PRs) with 95% confidence intervals (CIs). These analyses were treated as exploratory, non-inferential internal contrasts within the dataset of already retracted SR/MAs. Among 69,163 retracted publications in the Retraction Watch dataset, 1,071 (1.5%) were identified as retracted SR/MAs. Of those, 191 (17.8%) were paper mill-associated and 880 (82.2%) were not. In the database country field, China was recorded for 732/1,071 retracted SR/MAs (68.3%) and 177/191 SR/MAs with reported paper mill involvement (92.7%). The annual number of paper mill-associated SR/MAs peaked in 2023 (n=152). Within this restricted dataset, paper mill-associated SR/MAs had a longer median time to retraction (511 vs 459 days; p=0.024), more frequent results-related concerns (89.0% vs 30.0%), data issues (53.4% vs 17.5%), referencing concerns (53.9% vs 13.6%), and ethical concerns (8.9% vs 2.6%) (all p<0.001), and less frequent multi-country authorship (8.9% vs 17.6%; p=0.004). In univariable Poisson regression analyses restricted to retracted SR/MAs, larger observed proportions of reported paper mill involvement were seen among records linked to China in the database country field (PR 5.86, 95% CI 3.45-9.93), more recent publication years (PR 1.20 per 1-year increase, 95% CI 1.16-1.23), the medical broad domain (PR 3.66, 95% CI 1.66-8.08), and records with results-related concerns (PR 11.88, 95% CI 7.68-18.39). These PRs should not be interpreted as likelihoods, risks, or rates beyond the analyzed Retraction Watch subset. In this Retraction Watch-based analysis, reported paper mill involvement was identified in 191/1,071 retracted SR/MAs (17.8%). These records were most often linked to China in the database country field (177/191; 92.7%) and clustered in recent years, particularly 2023. These findings should be interpreted as descriptive patterns within detected and retracted records, not as country-level rates, comparative performance indicators, or estimates of the true prevalence of paper mill activity.