Skip to main content
SpringerLink
Log in
Book cover

New Trends in Earth-Science Outreach and Engagement pp 223–234Cite as

Big Benefit from Big Data: A Real-Time Data Product Creation and Distribution System

  • Chapter
  • First Online:

  • 1242 Accesses

Part of the book series: Advances in Natural and Technological Hazards Research ((NTHR,volume 38))

Abstract

New software systems are needed to meet the challenge of quickly transforming huge amounts of dynamic, real-time big data into accessible and useful information. The Information Management System (IMS) developed for the US Environmental Protection Agency’s AirNow program, which informs the public about current and predicted local air quality conditions, provides an efficient and flexible way to automatically ingest and process data to create useful information products. Although IMS was developed to manage US air quality data, it can easily be adapted for international use and to other scientific disciplines.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Converting individual data points to a gridded data file makes the data easier to display and is a preliminary step to creating map files. Algorithms are used to weigh and calculate values for the data points that are available near the fixed points of a regular, rectangular grid.

  2. 2.

    Shapefile: a common file format for showing spatial data.

  3. 3.

    Through performance testing and research, we determined that it is most efficient for each core of a CPU to build a single file product at a time. IMS servers are thus set by default to simultaneously build as many files as they have CPU cores.

Acknowledgments

 The authors wish to acknowledge the contributions of Jeffrey Prouty, Eric Gray, Patrick Zahn, Jennifer DeWinter, Alex Dove, and Mary Jo Teplitz of Sonoma Technology, Inc. (STI); Daniel Pryden, Judd Reed, and Liron Yahdav, formerly of STI; Scott Jackson of the US Environmental Protection Agency; and the US National Aeronautics and Space Administration for funding the AirNow Satellite Data Processor (ASDP).

Author information

Authors and Affiliations

  1. Sonoma Technology, Inc. (STI), 1455 N. McDowell Blvd, Petaluma, CA, 94954, USA

    Michael D. Haderman BS, Adam N. Pasch Ph.D., Timothy S. Dye MS, Alan C. Chan MS & Dianne S. Miller MS

  2. Environmental Protection Agency, 109 T.W. Alexander Drive, Research Triangle Park, Durham, NC, 27709, USA

    John E. White BS & Philip G. Dickerson BS

Authors
  1. Michael D. Haderman BS
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adam N. Pasch Ph.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Timothy S. Dye MS
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alan C. Chan MS
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dianne S. Miller MS
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. John E. White BS
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Philip G. Dickerson BS
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael D. Haderman BS .

Editor information

Editors and Affiliations

  1. College of Liberal Arts, The University of Findlay, Findlay, Ohio, USA

    Jeanette L. Drake

  2. Scenarios Network for Alaska and Arctic Planning (SNAP), International Arctic Research Center (IARC), University of Alaska-Fairbanks, Fairbanks, Alaska, USA

    Yekaterina Y. Kontar

  3. Education and Biology, The University of Findlay, Findlay, Ohio, USA

    Gwynne S. Rife

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Haderman, M.D. et al. (2014). Big Benefit from Big Data: A Real-Time Data Product Creation and Distribution System. In: Drake, J., Kontar, Y., Rife, G. (eds) New Trends in Earth-Science Outreach and Engagement. Advances in Natural and Technological Hazards Research, vol 38. Springer, Cham. https://doi.org/10.1007/978-3-319-01821-8_16

Download citation

Publish with us

Policies and ethics

Search

Navigation