Abstract
Background
Traditional site-monitoring techniques are not optimal in finding data fabrication and other nonrandom data distributions with the greatest potential for jeopardizing the validity of study results. TransCelerate BioPharma conducted an experiment testing the utility of statistical methods for detecting implanted fabricated data and other signals of noncompliance.
Methods
TransCelerate tested statistical monitoring on a data set from a chronic obstructive pulmonary disease (COPD) clinical study with 178 sites and 1554 subjects. Fabricated data were selectively implanted in 7 sites and 43 subjects by expert clinicians in COPD. The data set was partitioned to simulate studies of different sizes. Analyses of vital signs, spirometry, visit dates, and adverse events included distributions of standard deviations, correlations, repeated values, digit preference, and outlier/inlier detection. An interpretation team, including clinicians, statisticians, site monitoring, and data management, reviewed the results and created an algorithm to flag sites for fabricated data.
Results
The algorithm identified 11 sites (19%), 19 sites (31%), 28 sites (16%), and 45 sites (25%) as having potentially fabricated data for studies 2A, 2, 1A, and 1, respectively. For study 2A, 3 of 7 sites with fabricated data were detected, 5 of 7 were detected for studies 2 and 1A, and 6 of 7 for study 1. Except for study 2A, the algorithm had good sensitivity and specificity (>70%) for identifying sites with fabricated data.
Conclusions
We recommend a cross-functional, collaborative approach to statistical monitoring that can adapt to study design and data source and use a combination of statistical screening techniques and confirmatory graphics.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Baigent C, Harrell FE, Buyse M, Emberson JR, Altman DG. Ensuring trial validity by data quality assurance and diversification of monitoring methods. Clin Trials. 2008;5:49–55.
US Food and Drug Administration. Guidance for industry oversight of clinical investigations: a risk-based approach to monitoring. http://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/ucm269919. Accessed April 18, 2014.
Buyse M, George SL, Evans S, et al. The role of biostatistics in the prevention, detection and treatment of fraud in clinical trials. Stat Med. 1999;18:3435–3451.
Venet D, Doffagne E, Burzykowski T, et al. A statistical approach to central monitoring of data quality in clinical trials. Clin Trials. 2012:9:705–713.
Lindblad AS, Manukyan Z, Purohit-Sheth T, et al. Central site monitoring: results from a test of accuracy in identifying trials and sites failing Food and Drug Administration inspection. Clin Trials. 2014;11:205–217.
Kirkwood AA, Cox T, Hackshaw A. Application of methods for central statistical monitoring in clinical trials. Clin Trials. 2013;10:783–806.
Knepper D, Fenske C, Nadolny P, et al. Detecting data quality issues in clinical trials: current practices and recommendations. Therapeutic Innovation & Regulatory Science. 2016;50:15–21.
O’Kelly M. Using statistical techniques to detect fraud: a test case. Pharm Stat. 2004:3:237–246.
Pogue JM, Devereaux PJ, Thurland K, et al. Central statistical monitoring: detecting fraud in clinical trials. Clin Trials. 2013:10:225–235.
Wu X, Carlsson M. Detecting data fabrication in clinical trials from cluster analysis perspective. Pharm Stat. 2011;10:257–264.
Garrett M, Fitzmaurice N, Laird M, Ware J. Applied Longitudinal Analysis. New York, NY: John Wiley & Sons Inc; 2005.
Mahalanobis PC. On the generalised distance in statistics. Proc Natl Inst Sci (Calcutta). 1936;2:49–55.
Murrel P. R Graphics. Boca Raton, FL: Chapman and Hall/CRC; 2006.
Heath D. Effective graphics made simple using SAS/GRAPH® SG procedures. Paper presented at: SAS Global Forum 2008, March 16–19, 2008, San Antonio, TX. http://www2.sas.com/proceedings/forum2008/255-2008.pdf.
Wegman EJ. Hyperdimensional data analysis using parallel coordinates. J Am Stat Assoc. 1990;85:664–675.
Gough J, Wilson B, Zerola M, et al. Defining a central monitoring capability: sharing the experience of TransCelerate BioPharma’s approach, part 2. Therapeutic Innovation & Regulatory Science. 2016;50:8–14.
Author information
Authors and Affiliations
Corresponding author
Additional information
Author’s Note
Anne S. Lindblad, Gaurav Sharma, Gary R. Gensler, Zorayr Manukyan, Abigail G. Matthews, and Yodit Seifu also are members of the interpretation team.
Rights and permissions
This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Knepper, D., Lindblad, A.S., Sharma, G. et al. Statistical Monitoring in Clinical Trials: Best Practices for Detecting Data Anomalies Suggestive of Fabrication or Misconduct. Ther Innov Regul Sci 50, 144–154 (2016). https://doi.org/10.1177/2168479016630576
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1177/2168479016630576