Abstract
Editorial procedures in the social and biomedical sciences are said to promote studies that falsely reject the null hypothesis. This problem may also exist in major marketing journals. Of 692 papers using statistical significance tests sampled from theJournal of Marketing, Journal of Marketing Research, andJournal of Consumer Research between 1974 and 1989, only 7.8% failed to reject the null hypothesis. The percentage of null results declined by one-half from the 1970s to the 1980s. TheJM and theJMR registered marked decreases. The small percentage of insignificant results could not be explained as being due to inadequate statistical power.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Angell, Marcia. (1989). “Negative Studies,”New England Journal of Medicine 321, 464–466.
Atkinson, Donald R., Michael J. Furlong, and Bruce E. Wampold. (1982). “Statistical Significance, Reviewer Evaluations, and the Scientific Process: Is There a (Statistically) Significant Relationship?”Journal of Counseling Psychology 29, 189–194.
Bakan, David. (1966). “The Test of Significance in Psychological Research,”Psychological Bulletin 77, 423–437.
Bozarth, Jerold D., and Ralph R. Roberts, Jr. (1972). “Signifying Significant Significance,”American Psychologist 27, 774–775.
Cohen, Jacob. (1962). “The Statistical Power of Abnormal-Social Psychological Research: A Review,”Journal of Abnormal and Social Psychology 65, 145–153.
Cohen, Jacob. (1988).Statistical Power Analysis for the Behavioral Sciences. Hillsdale, N.J.: Lawrence Erlbaum Associates.
Cohen, Jacob. (1990). “Things I Have Learned (So Far),”American Psychologist 45, 1304–1312.
Coursol, Allan, and Edwin E. Wagner. (1986). “Effect of Positive Findings on Submission and Acceptance Rates: A Note on Meta-Analysis Bias,”Professional Psychology: Research and Practice 17, 136–137.
Dickersin, K., S. Chan, T. C. Chalmers, H. S. Sacks, and H. Smith, Jr. (1987). “Publication Bias and Clinical Trials,”Controlled Clinical Trials 8, 343–353.
Fagley, N. S. (1985). “Applied Statistical Power Analysis and the Interpretation of Nonsignificant Results by Research Consumers,”Journal of Counseling Psychology 32, 391–396.
Feige, Edgar L. (1975). “The Consequences of Journal Editorial Policies and a Suggestion for Revision,”Journal of Political Economy 83, 1291–1295.
Greenwald, Anthony G. (1975). “Consequences of Prejudice Against the Null Hypothesis,”Psychological Bulletin 82, 1–20.
Kerr, Steven, James Tolliver, and Doretta Petree. (1977). “Manuscript Characterisics Which Influence Acceptance for Management and Social Science Journals,”Academy of Management Journal 20, 132–141.
Kupfersmid, Joel, and Michael Fiala. (1991). “A Survey of Attitudes and Behaviors of Authors Who Publish in Psychology and Education Journals,”American Psychologist 46, 249–250.
Rosenthal, Robert. (1979). “The ‘File Drawer Problem’ and Tolerance for Null Results,”Psychological Bulletin 86, 638–641.
Rosnow, Ralph L., and Robert Rosenthal. (1989). “Statistical Procedures and the Justification of Knowledge in Psychological Science,”American Psychologist 44, 1276–1284.
Rourke, Byron P., and Louis Costa. (1979). “Editorial Policy II,”Journal of Clinical Neuropsychology 1, 93–95.
Rowney, Julie A., and Thomas J. Zenisek. (1980). “Manuscript Characteristics Influencing Reviewers' Decisions,”Canadian Psychology 21, 17–21.
Rust, Roland T., Donald R. Lehmann, and John U. Farley. (1990). “Estimating Publication Bias in Meta-Analysis,”Journal of Marketing Research 27, 220–226.
Salsburg, David S. (1985). “The Religion of Statistics as Practiced in Medical Journals,”American Statistician 39, 220–223.
Sawyer, Alan G., and A. Dwayne Ball. (1981). “Statistical Power and Effect Size in Marketing Research,”Journal of Marketing Research 18, 275–290.
Shaddish, William R., Maria Doherty, and Linda M. Montgomery. (1989). “How Many Studies Are in the File Drawer? An Estimate from the Family/Marital Psychotherapy Literature,”Clinical Psychology Review 9, 589–603.
Simes, Robert J. (1986). “Publication Bias: The Case for an International Registry of Clinical Trials,”Journal of Clinical Oncology 4, 1529–1541.
Smart, Reginald G. (1964). “The Importance of Negative Results in Psychological Research,”Canadian Psychologist 5, 225–232.
Smith, Mary L. (1980). “Publication Bias and Meta-Analysis,”Evaluation in Education 4, 22–24.
Sommer, Barbara. (1987). “The File Drawer Effect and Publication Rates in Menstrual Cycle Research,”Psychology of Women Quarterly 11, 233–241.
Sterling, Theodore D. (1959). “Publication Decisions and their Possible Effects on Inferences Drawn from Tests of Significance — or Vice Versa,”Journal of the American Statistical Association 54, 30–34.
Author information
Authors and Affiliations
Additional information
The authors thank Richard Bagozzi, Gilbert Churchill, James Engel, John Farley, Anthony Greenwald, Daniel Vetter, the editor, and two anonymous reviewers for comments on earlier versions of this paper. Any errors remaining are our responsibility.
Rights and permissions
About this article
Cite this article
Hubbard, R., Armstrong, J.S. Are null results becoming an endangered species in marketing?. Marketing Letters 3, 127–136 (1992). https://doi.org/10.1007/BF00993992
Issue Date:
DOI: https://doi.org/10.1007/BF00993992