Skip to main content
SpringerLink
Log in

Bias in toxicology

  • Regulatory Toxicology
  • Published:
Archives of Toxicology Aims and scope Submit manuscript

Abstract

The potential for bias, i.e., influences that cause results to deviate systematically from the truth is substantial both in toxicological research and in the performance of standardized toxicological testing. In this contribution, major potential sources of bias in toxicological research and testing are identified. Due to the lack of empirical studies of bias in toxicology, very little is known about its prevalence and impact. Areas to consider for such studies are pointed out, and it is suggested that such investigations should be given priority.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. This definition has been adopted by, among others, Feinleib (1987), Fletcher and Fletcher (2005) and Sackett (1979). Definitions used by others have very similar wordings; Silman and MacFarlane (2002), for instance, define bias as “a systematic deviation from the truth” (p. 201).

  2. It should be noted that in cases where research or toxicity testing is performed with the aim to enable a human health risk assessment there would be a potential for a biased choice of experimental model: more often than not important information about species differences in sensitivity towards a specific exposure is missing, and therefore we have a restricted understanding of the extent to which the animal model is representative of humans. Sometimes the animal model will overestimate human toxicity (creating a false positive result), and sometimes it will be less sensitive than humans (creating a false negative result). In both of these cases a bias has been introduced in the process, given the ultimate aim of a reliable human health risk assessment. This issue, however, is more pertinent to a discussion of possible biases in risk assessment (rather than in toxicology), so we will not pursue it further in the present contribution.

  3. For analyses of how to manage conflicts of interest, see, e.g., Shamoo and Resnik (2003), OECD (2003) and OECD (2005). A critical discussion of measures taken to manage conflicts of interest in toxicology is found in Maurissen et al. (2005).

  4. Standardized testing may still of course be supplemented with determination of certain mechanistic data in order to adjust the testing procedure or to facilitate the interpretation of data.

  5. For examples from the EU, see for instance: Council Directive 67/548 and 92/32/EEC defining the base-set of data required for risk assessment; Council Regulation 793/93/EEC defining how data reporting should be conducted; Directive 2001/83/EC and Commission Directive 2003/63/EC regulating the safety assessment of pharmaceuticals for human use; Council Directive 91/414/EEC concerning plant protection products; Council Directive 76/768/EEC concerning cosmetic products; Council Directive 89/107/EEC concerning food additives; Directive 98/8/EC concerning biocides. In other jurisdictions corresponding regulation can be found.

  6. The total number of government-funded studies was 119, and the total number of industry-funded studies was 11.

  7. See for instance the OECD test guidelines for carcinogenicity testing (no. 451), for chronic toxicity testing (no. 452) or for the two-generation reproductive toxicity test (no. 416).

  8. Obviously, whether a result is interpreted as significant or not also depends on which confidence interval is used. However, that choice is almost always governed by conventions, and does not in practice differ between studies.

  9. The absolute probability of a false positive is equal to p(R& H0) =  p(H0)·p(R|H0) where H0 is the null hypothesis and R the outcome of the experiment.

  10. Studies carried out under GLP are subject to inspection by an internal quality assurance programme, which is, among other things, responsible for performing an audit of raw data and the final report, ensuring that the data are complete and that they have be accurately reported (OECD Series on Principles of GLP and Compliance Monitoring, no. 4).

  11. This is possible when investigating publication bias within medical research involving human subjects because such research must be approved by an authority (usually an ethics committee) before being carried out. For examples of such investigations, see Easterbrook et al. (1991) and Stern and Simes (1997).

  12. In the US, data from studies that are funded by federal agencies must be made available to the public. For a critical discussion of this, see Crissman et al. (1999).

References

  • Barnes DE, Bero LA (1998) Why review articles on the health effects of passive smoking reach different conclustions. JAMA 279(19):1566–1570

    Article  PubMed  CAS  Google Scholar 

  • Bekelman JE, Li Y, Gross CP (2003) Scope and impact of financial conflicts of interest in biomedical research. JAMA 289(4):454–464

    Article  PubMed  Google Scholar 

  • Bero LA, Rennie D (1996) Influences on the quality of published drug studies. Int J Technol Assess Health Care 12(2):209–237

    Article  PubMed  CAS  Google Scholar 

  • Boorman GA, Haseman JK, Waters MD, Hardisty JF, Sills RC (2002) Quality review procedures necessary for rodent pathology databases and toxicogenomic studies: The National Toxicology Program Experience. Toxicol Pathol 30(1):88–92. doi: 10.1080/01926230252824752

    Article  PubMed  Google Scholar 

  • Commission Directive 2003/63/EC on Community code relating to medicinal products for human use. Official Journal no. L 159, 27/06/2003, pp 46–94

  • Correspondence about publication ethics and regulatory toxicology and pharmacology (2003). Int J Occup Environ Health 9(4):386–391 http://www.ijoeh.com/pfds/0904_Correspondence.pdf

    Google Scholar 

  • Council Directive 76/768/EEC (1976) on the approximation of the laws of the Member States relating to cosmetic products. Official Journal no-.L 262, 27/09/1976 pp. 169–200

    Google Scholar 

  • Council Directive 89/107/EEC (1988) on the approximation of the laws of the Member States concerning food additives authorized for use in foodstuffs intended for human consumption Official Journal no. L 40, 11/02/1989, pp 27–33

  • Council Directive 91/414/EEC concerning the placing of plant protection products on the market (1991) Official Journal no. L 230, 19/08/1991, pp 1–32

  • Council Directive 92/32/EEC (1992) Amending for the seventh time Directive 67/548/EEC on the approximation of the laws, regulations and administrative provisions relating to the classification, packaging and labelling of dangerous substances. Official Journal, no. L 154, 05/06/1992, pp. 1–29

    Google Scholar 

  • Council of the European Union (2006) Presidency Proposal for a regulation of the European Parliament and of the council concerning the Registration, Evaluation, Authorisation and Restrictions of Chemicals (REACH), establishing a European Chemicals Agency and amending Directive 1999/45/EC and Regulation (EC) on Persistent Organic Pollutants. 15921/1/05 REV 1, Brussels, 9 March 2006, available at http://www.register.consilium.eu.int

  • Council Regulation (EEC), no. 793/93 (1993) on the evaluation and control of the risks of existing substances. Official Journal, no L 84, 05/04/1993, pp 1–75

    Google Scholar 

  • Council Directive 67/548/EEC on the approximation of laws, regulations and administrative provisions relating to the classification, packaging and labelling of dangerous substances (1967) Official Journal no. 196, 16/08/1967, pp 1–98

  • Crissman JW, Bus JR, Miller RR(1999) Toxicology: judge data or dollars? Environ Health Perspect 107(10): A489–491

    Article  PubMed  CAS  Google Scholar 

  • Crissman JW, Goodman DG, Hildebrandt PK, Maronpot RR, Prater DA, Riley JH, Seaman WJ, Thake DC (2004) Best practices guideline: toxicologic histopathology. Toxicol Pathol 32:126–131. doi: 10.1080/01926230490268756

    Article  PubMed  CAS  Google Scholar 

  • Cunningham AR, Rosenkranz HS (2001) Estimating the extent of the health hazard posed by high-production volume chemicals. Environ Health Perspect 109(9):953–956

    Article  PubMed  CAS  Google Scholar 

  • DeMets DL (1999) Statistics and ethics in medical research. Sci Eng Ethics 5(1):97–117

    Article  PubMed  Google Scholar 

  • Dickersin K (1990) The existence of publication bias and risk factors for its occurrence. JAMA 268(10):1385–1389

    Article  Google Scholar 

  • Dickersin K (1997) How important is publication bias? A synthesis of available data. AIDS Educ Prev 9(suppl A):15–21

    PubMed  CAS  Google Scholar 

  • Dickersin K, Rennie D (2003) Registering clinical trials. JAMA 290(4):516–523

    Article  PubMed  Google Scholar 

  • Directive 98/8/EC of the European Parliament and of the Council (1998) concerning the placing of biocidal products on the market. Official Journal of the European Communities, L123, 24/04/1998, pp. 1–63

  • Directive 2001/20/EC of The European Parliament and of the Council (2001). Official Journal of the European Communities no. L 121, 01/05/2001, pp. 34–44

  • Directive 2001/83/EC of the European Parliament and of the Council on Community code relating to medicinal products for human use (2001) Official Journal of the European Communities no. L 311, 28/11/2001, pp 67–128

  • Djulbegovic B, Lacevic M, Cantor A, Fields KK, Bennett CL, Adams JR, Kuderer NM, Lyman GH (2000) The uncertainty principle and industry-sponsored research. Lancet 356:635–638. doi: 10.1016/S0140-6736(00)02605-2

    Article  PubMed  CAS  Google Scholar 

  • Doucet MS, Bridge DA, Grimlund RA, Shamoo AE (1994) An application of stratified sampling techniques for research data. Account Res 3:237–247

    Google Scholar 

  • Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR (1991) Publication bias in clinical research. Lancet 337(8746):867–872

    Article  PubMed  CAS  Google Scholar 

  • European Commission (2003). Technical Guidance Document in support of Commission Directive 93/67/EEC on Risk Assessment for new notified substances, Commission Regulation (EC) No 1488/94 on Risk Assessment for existing substances and Directive 98/8/EC of the European Parliament and of the Council concerning the placing of biocidal products on the market. Part 2

  • Feinleib M (1987) Biases and weak associations. Prev Med 16:150–164

    Article  PubMed  CAS  Google Scholar 

  • Fletcher RW, Fletcher SW (2005) Clinical epidemiology: the essentials, 4th edn. Lippincott Williams & Wilkins, Baltimore

    Google Scholar 

  • GAO (2005). Chemical regulation: options exist to improve EPA’s ability to assess health risks and manage its chemical review program. Report to Congressional Requesters, GAO-05-458. (http://www.gao.gov/cgi-bin/getrprt?GAO-05-458)

  • Godlee F, Dickerson K (2003) Bias, subjectivity, chance and conflict of interest in editorial decisions. In: Godlee F (ed) Peer review in health sciences, 2nd edn. BMJ Publishing Group, London, pp 91–117

    Google Scholar 

  • Hansson SO, Rudén C (2006a) Evaluating the risk decision process. Toxicology 218(2–3):100–111

    Article  PubMed  CAS  Google Scholar 

  • Hansson SO, Rudén C (2006b) Priority setting in the REACH system. Toxicol Sci 90(2):304–308. doi: 10.1093/toxsci/kfj071

    Article  PubMed  CAS  Google Scholar 

  • Hansson SO, Rudén C (2007) Towards a theory of tiered testing. Regul Toxicol Pharmacol (in press)

  • Holland T (2001) A survey of discriminant methods used in toxicological histopathology. Toxicol Pathol 29(2):269–273

    Article  PubMed  CAS  Google Scholar 

  • Holland T (2005) The comparative power of the discriminant methods used in toxicological pathology. Toxicol Pathol 33:490–494. doi: 10.1080/01926230590965382

    Article  PubMed  Google Scholar 

  • IARC Monographs on the evaluation of carcinogenic risks to humans (1980) Supplement 2

  • Kaptchuk TJ (2003) Effect of interpretive bias on research evidence. BMJ 326(7404):1453–1455. doi: 10.1136/bmj.326.7404.1453

    Article  PubMed  Google Scholar 

  • Koehler JJ (1993) The influence of prior beliefs on scientific judgments of evidence quality. Organ Behav Hum Decis Process 56:28–55

    Article  Google Scholar 

  • Lexchin J, Bero LA, Djulbegovic B, Clark O (2003) Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ 326(7400):1167–1170. doi: 10.1136/bmj.326.7400.1167

    Article  PubMed  Google Scholar 

  • Lynn FM (1986) The interplay of science and values in assesing and regulating environmental risks. Sci Technol Hum Values 11(2):40–50

    Google Scholar 

  • Maurissen JP, Gilbert SG, Sander M, Beauchamp TL, Johnson S, Schwetz BA, Goozner M, Barrow CS (2005) In: Workshop proceedings: managing conflict of interest in science. A little consensus and a lot of controversy. Toxicol Sci 87(1) 11–14. doi: 10.1093/toxsci/kfi240

  • Maxwell C (1981) Clinical trials, reviews, and the journal of negative results. Br J Clin Pharmacol 1:15–18

    Google Scholar 

  • Murphy EA (1978) The logic of medicine. The John Hopkins University Press, Baltimore

    Google Scholar 

  • Nordberg A, Rudén C (2007) The usefulness of the bioconcentration factor as a tool for priority setting in chemicals control. Toxicol Lett 168:113–120. doi: 10.1016/j.toxlet.2006.11.004

    Article  PubMed  CAS  Google Scholar 

  • OECD guidelines for the testing of chemicals no. 451 carcinogenicity studies (adopted 12 May 1981)

  • OECD guidelines for the testing of chemicals no. 452 chronic toxicity studies (adopted 12 May 1981)

  • OECD Guidelines for the testing of chemicals no. 416 two-generation reproduction toxicity study (Updated Guideline, adopted 22 January 2001)

  • OECD (2003) Managing conflict of interest in the public service. OECD guidelines and country experiences

  • OECD (2005) Managing conflict of interest in the public sector. A toolkit

  • OECD Series on principles of good laboratory practice and compliance monitoring, no. 2 (1995) Guidance for GLP Monitoring Authorities. OCDE/GD(95)66

  • OECD series on principles of good laboratory practice and compliance monitoring, no. 1 (1998) OECD principles on good laboratory practice (as revised in 1997). ENV/MC/CHEM(98)17

  • Prasse K, Hildebrandt P, Dodd D (1986) Letter to the editor. Vet Pathol 23:540–541

    Article  PubMed  CAS  Google Scholar 

  • Purchase IFH (2004) Fraud, errors and gamesmanship in experimental toxicology. Toxicology 202:1–20. doi: 10.1016/j.tox.2004.06.029

    Article  PubMed  CAS  Google Scholar 

  • Resnik DB (2000a) Financial interests and research bias. Perspect Sci 8(3):255–285. doi: 10.1162/106361400750340497

    Article  Google Scholar 

  • Resnik DB (2000b) Statistics, ethics and research: an agenda for education and reform. Account Res 8:163–188

    Article  Google Scholar 

  • Rice JM (2005) The carcinogenicity of acrylamide. Mutat Res 580:3–20. doi: 10.1016/j.mrgentox.2004.09.008

    PubMed  CAS  Google Scholar 

  • Roberts I, Kwan I, Evans P, Haig, S (2002) Does animal experimentation inform human healthcare? Observations from a systematic review of international animal experiments on fluid resucitation. BMJ 324:474–476

    Article  PubMed  Google Scholar 

  • Rothman K.J (1991) The ethics of research sponsorship. J Clin Epidemiol 44(Suppl 1):S25–S28

    Article  PubMed  Google Scholar 

  • Rudén C (2001) Interpretations of primary carcinogenicity data in 29 trichloroethylene risk assessments. Toxicology 169(3):209–225. doi: 10.1016/S0300-483X(01)00525-X

    Article  PubMed  Google Scholar 

  • Sackett DL (1979) Bias in Analytic Research. J Chron Dis 32:51–63

    Article  PubMed  CAS  Google Scholar 

  • Shamoo AE, Resnik DB (2003) Responsible conduct of research. Oxford University Press, New York

    Google Scholar 

  • Silman A, MacFarlane GJ (2002) Epidemiological studies. A practical guide, 2nd edn. Cambridge University Press, London

    Google Scholar 

  • Sterling TD (1959) Publication decisions and their possible effects on inferences drawn from tests of significance—or vice versa. J Am Stat Assoc 54(285):30–34

    Article  Google Scholar 

  • Stern JM, Simes R J (1997) Publication bias: evidence of delayed publication in a cohort study of clinical research projects. BMJ 315(7109):640–645

    PubMed  CAS  Google Scholar 

  • vom Saal FS, Welshons WV (2006) Large effects from small exposures. II. The importance of positive controls in low-dose research on bisphenol A. Environ Res 100:50–76. doi: 10.1016/j.envres.2005.09.001s

    Article  PubMed  CAS  Google Scholar 

  • Weinberg AM (1972) Science and trans-science. Minerva 10:208–222. doi: 10.1007/BF01682418

    Google Scholar 

  • Whitbeck C (2004) Trust and the future of research. Phys Today 57(11):48–53

    Article  Google Scholar 

Download references

Acknowledgements

This work was funded by the Swedish Research Council for Environment, Agricultural Sciences and Spatial planning (FORMAS), and the Swedish Foundation for Strategic Environmental Research (MISTRA). An earlier version of this paper was presented at a seminar hosted by professor Helen Håkansson at the Institute for Environmental Medicine, Karolinska Institutet. The authors wish to thank participants in that seminar for helpful comments.

Author information

Authors and Affiliations

  1. Department of Philosophy and the History of Technology, Royal Institute of Technology (KTH), Teknikringen 78 B, 100 44, Stockholm, Sweden

    Birgitte Wandall, Sven Ove Hansson & Christina Rudén

Authors
  1. Birgitte Wandall
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sven Ove Hansson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christina Rudén
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Birgitte Wandall.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wandall, B., Hansson, S.O. & Rudén, C. Bias in toxicology. Arch Toxicol 81, 605–617 (2007). https://doi.org/10.1007/s00204-007-0194-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00204-007-0194-5

Keywords

Search

Navigation