Abstract
This paper presents a scan statistic, progressive upper level set (PULSE) scan statistic, for geospatial hotspot detection and its software implementation. Like ULS, the PULSE scan statistic is based on the arbitrarily shaped scan window and can be adapted for a network setting. PULSE is a refinement of the upper level set (ULS) scan statistic. Like some other likelihood based scanning devices, the ULS scan statistic identifies maximum likelihood estimate (MLE) zones that tend to be ‘stringy’ and sprawling. Its search path increases possibility of inclusion of extraneous cells in its MLE zones and, to a smaller extent, of exclusion of cells that belong to a true hotspot from its MLE zone. The PULSE scan statistic achieves improvement over the ULS scan statistic in two ways. First, it begins its search for a most likely zone with a large population of candidate zones obtained by modifying the ULS tree structure and continues its search using a genetic algorithm. Secondly, to reduce chances of generating an MLE that is excessively stringy and that includes extraneous cells in the MLE zone, PULSE uses cardinality and compactness of zones along with their likelihoods as the fitness function in the genetic algorithm and uses several pertinent criteria including evenness of intra-zone cellular response ratios to determine the MLE zone. To reduce computation, Gumbel distribution of extreme values is used to determine the p-value of the MLE zone. Better results come at the cost of increased processing time. An evaluative performance study is presented.
Similar content being viewed by others
References
Abrams AM, Kulldorff M, Kleinman K (2006) Empirical/asymptotic p-values for Monte Carlo-based hypothesis testing: and application to cluster detection using the scan statistic. Adv Disease Surveill 1:1. http://www.isdsjournal.org/article/view/269/211
Biomedware (2001) Software for the environmental and health sciences. Biomedware, Ann Arbor
Duczmal L, Assunção RA (2004) A simulated annealing strategy for the detection of arbitrarily shaped spatial clusters. Comput Stat Data Anal 45: 269–286
Duczmal L, Kulldorff M, Huang L (2006) Evaluation of spatial scan statistics for irregular shaped clusters. J Comput Graph Stat 15: 428–442
Duczmal L, Cançado ALF, Takahashi RHC, Bessegato L (2007) A genetic algorithm for irregularly shaped spatial scan statistics. Comput Stat Data Anal 52: 43–52
Glaz J, Balakrishnan N (1999) Scan statistics and applications. Springer Publications, Netherlands
Glaz J, Naus J, Wallenstein S (2001) Scan statistics. Springer Publications, US
Kotz S, Nadarajah S (2000) Extreme value distributions: theory and applications. Imperial College Press, London
Kulldorff M (1997) A spatial scan statistic. Commun Stat Theory Methods 26(6): 1481–1496
Kulldorff M (2001) Prospective time-period geographical disease surveillance using a scan statistic. J R Stat Soc Ser A 164: 61–72
Kulldorff M (2006) SaTScanTM v 7.0: software for the spatial and space–time scan statistics. Information Management Services Inc, Silver Spring
Kulldorff M, Nagarwalla N (1995) Spatial disease clusters: detection and inference. Stat Med 14: 799–810
Kulldorff M, Rand K, Gherman G, Williams G, DeFrancesco D (1998) SaTScan v 2.1: software for the spatial and space-time scan statistics. National Cancer Institute, Bethesda
Kulldorff M, Huang L, Pickle L, Duczmal L (2006) An elliptic spatial scan statistic. Stat Med 25: 3929–3943
Mostashari F, Kulldorff M, Hartman J, Miller J, Kulasekera V (2003) Dead bird clustering: An early warning system for West Nile virus activity. http://www.cdc.gov/ncidod/eid/vol9no6/02-0794.htm
Patil GP, Taillie C (2004a) Upper level set scan statistic for detecting arbitrarily shaped hotspots. Environ Ecol Stat 11: 183–197
Patil GP, Taillie C (2004b) Multiple indicators, partially ordered sets, and linear extensions: multi-criterion ranking and prioritization. Environ Ecol Stat 11: 199–228
Patil GP, Acharya R, Glasmier A, Myers W, Phoha S, Rathbun S (2006a) Hotspot detection and prioritization geoinformatics for digital governance. In: Chen H, Brandt L, Gregg V, Traunmuller R, Dawes S, Hovy E, Macintosh A, Larson C (eds) Digital government: advanced research and case studies. Springer, US
Patil GP, Modarres R, Myers WL, Patankar P (2006b) Spatially constrained clustering and upper level set scan hotspot detection in surveillance geo informatics. Environ Ecol Stat 13: 365–377
Patil GP, Duczmal L, Haran M, Patankar P (2006c) On PULSE: the progressive upper level set scan statistic system for geospatial and spatiotemporal hotspot detection. In: The 7th annual international conference on digital government research, San Diego, CA
Patil GP, Patil VD, Pawde SP, Phoha S, Singhal V, Zambre R (2008) Digital governance, hotspot geoinformatics, and sensor networks for monitoring, etiology, early warning, and sustainable management. In: Joshi PK (eds) Geoinformatics for natural resource management. Nova Science Publishers, New York (in press)
Patil GP, Joshi SW, Myers WL, Koli RE (2009) ULS scan statistic for hotspot detection with continuous gamma response. In: Glaz J, Pozdnyakov V, Wallenstein S (eds) Scan statistics: methods and applications. Birkhäuser, Boston
Patil GP, Joshi SW, Koli RE (2009b) Digital governance and hotspot geoinfomatics with continuous fractional response. In: Indian statistical institute platinum jubilee volume on statistics. Kolkata, India (in press)
Tango T, Takahashi K (2005) A flexibly shaped spatial scan statistic for detecting clusters. Int J Health Geograph. http://www.i-healthgeographics.com/content/4/1/11
Wallenstein S (2009) Joseph Naus: Father of the scan statistic. In: Glaz J, Pozdnyakov V, Wallenstein S (eds) Scan statistics: methods and applications. Birkhauser, Boston
Waller L (2002) Methods for detecting disease clustering in time or space. In: Brookmeyer R, Stroup D (eds) Statistical methods and principles in public health surveillance. Oxford University Press, Oxford
Whitley D (1994) A genetic algorithm tutorial. http://www.cs.iastate.edu/~honavar/ga_tutorial.pdf
Author information
Authors and Affiliations
Corresponding author
Additional information
This material is based upon work supported by the United States National Science Foundation under Grant No. 0307010. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors(s) and do not necessarily reflect the views of the agencies.
Rights and permissions
About this article
Cite this article
Patil, G.P., Joshi, S.W. & Koli, R.E. PULSE, progressive upper level set scan statistic for geospatial hotspot detection. Environ Ecol Stat 17, 149–182 (2010). https://doi.org/10.1007/s10651-010-0140-1
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10651-010-0140-1