Abstract
The key challenge for Dynamic Data Driven Applications Systems (DDDAS) operating in time-varying environments is to predict when the learned model may lose relevance. If the learned model loses relevance, then the autonomous system is at risk of making wrong decisions. The Entropic Value at Risk (EVAR) is a computationally efficient and coherent risk measure that can be utilized to quantify this model relevance. The value of EVAR is calculated with respect to an assumed confidence value; e.g., a 90% confidence may be desired for robust decision-making. Without a model on the confidence value directly, there is no guarantee that EVAR calculations will reflect the uncertainty present in dynamic real-world environments. In this paper, we present a Bayesian model and learning algorithms to predict the state-dependent confidence necessary for calculating the EVAR in time-varying datasets. We discuss applications of the data-driven EVAR to a monitoring problem, in which a DDDAS agent has to chose a set of sensing locations in order to maximize the expected EVAR of the acquired data. In this way, the DDDAS agent can learn a model on an underlying phenomenon of interest by prioritizing the areas where the model is most likely incorrect but highly valued. We empirically demonstrate the efficacy of the presented model and learning algorithms on five real-world datasets. We show that, overall, the EVAR-Real-time Adative Prediction of Time-varying and Obscure Rewards (EVAR-RAPTOR) algorithm outperforms EVAR-Predicted Information Gain* (EVAR-PIG*) as well as naive searches such as random and sequential search across these five real-world datasets.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
F. Darema, Dynamic data driven applications systems: a new paradigm for application simulations and measurements, in International Conference on Computational Science, Krakow (Springer, 2004), pp. 662–669 https://link.springer.com/book/10.1007/b97988
F. Darema, Grid computing and beyond: the context of dynamic data driven applications systems. Proc. IEEE 93(3), 692–697 (2005)
F. Darema, DDDAS, a key driver for large-scale-big-data and large-scale-big-computing. Proc. Comput. Sci. 51, 2463 (2015)
A. Hobson, A new theorem of information theory. J. Stat. Phys. 1(3), 383–391 (1969)
A. Ahmadi-Javid, Entropic value-at-risk: a new coherent risk measure. J. Optim. Theory Appl. 155(3), 1105–1123 (2012)
R.T. Rockafellar, S. Uryasev, Optimization of conditional value-at-risk. J. Risk 2, 21–42 (2000)
T. Kim, A.V. Nefian, M.J. Broxton, Photometric recovery of ortho-images derived from apollo 15 metric camera imagery, in Advances in Visual Computing (Springer, 2009), pp. 700–709
T. Kim, A.V. Nefian, M.J. Broxton, Photometric recovery of apollo metric imagery with lunar-lambertian reflectance. Electron. Lett. 46(9), 631–633 (2010)
D.Y. Little, F.T. Sommer, Learning and exploration in action-perception loops. Front. Neural Circuits 7, 1–19 (2013)
A.M. Axelrod, Learning to exploit time-varying heterogeneity in distributed sensing using the information exposure rate. Master’s thesis, Oklahoma State University, 2015
C.C. Heyde, E. Seneta, Studies in the history of probability and statistics. XXXI. The simple branching process, a turning point test and a fundamental inequality: a historical note on I.J. bienaymé. Biometrika 59(3), 680–683 (1972)
D. Applebaum, Lévy processes: from probability to finance and quantum groups. Not. AMS 51(11), 1336–1347 (2004)
M. Tokic, Adaptive ε-greedy exploration in reinforcement learning based on value differences, in KI 2010: Advances in Artificial Intelligence (Springer, 2010), pp. 203–210
W. Jouini, D. Ernst, C. Moy, J. Palicot, Upper confidence bound based decision making strategies and dynamic spectrum access, in 2010 IEEE International Conference on Communications (ICC), Cape Town (IEEE, 2010), pp. 1–5 http://icc2010.ieee-icc.org/
A. Carpentier, A. Lazaric, M. Ghavamzadeh, R. Munos, P. Auer, Upper-confidence-bound algorithms for active learning in multi-armed bandits, in Algorithmic Learning Theory (Springer, 2011) pp. 189–203
C. Gehring, D. Precup, Smart exploration in reinforcement learning using absolute temporal difference errors, in Proceedings of the 2013 International Conference on Autonomous Agents and Multi-agent Systems, Saint Paul. International Foundation for Autonomous Agents and Multiagent Systems, 2013, pp. 1037–1044 https://dl.acm.org/citation.cfm?id=2484920
D. Russo, B. Van Roy, An information-theoretic analysis of Thompson sampling. arXiv preprint arXiv:1403.5341 (2014)
J.C. Principe, D. Xu, Information-theoretic learning using Renyis quadratic entropy, in Proceedings of the First International Workshop on Independent Component Analysis and Signal Separation, Aussois, France, 1999, pp. 407–412 https://scholar.googleusercontent.com/scholar.bib?q=info:IFH_36rOMQgJ:scholar.google.com/&output=citation&scisig=AAGBfm0AAAAAW3GA7DJPrAxvpEvPRhC_z9ovoHvVeTWe&scisf=4&ct=citation&cd=-1&hl=en&scfhb=1
P. Reverdy, R.C. Wilson, P. Holmes, N.E. Leonard, Towards optimization of a human-inspired heuristic for solving explore-exploit problems, in CDC, Maui, 2012, pp. 2820–2825 http://www.ieeecss.org/CAB/conferences/cdc2012/
D. Ryabko, Time-series information and learning, in 2013 IEEE International Symposium on Information Theory Proceedings (ISIT), Istanbul (IEEE, 2013), pp. 1392–1395 http://www.proceedings.com/19451.html
S.G. Nora Ayanian, Persistent monitoring of stochastic spatio-temporal phenomena with a small team of robots, in Proceedings of Robotics: Science and Systems, Berkeley, California, USA, July 2014 http://rll.berkeley.edu/RSS2014/
D. Russo, B. Van Roy, Learning to optimize via information-directed sampling, in Advances in Neural Information Processing Systems, Montreal, 2014, pp. 1583–1591 https://nips.cc/Conferences/2014/
J.M. Hernández-Lobato, M.A. Gelbart, R.P. Adams, M.W. Hoffman, Z. Ghahramani, A general framework for constrained bayesian optimization using information-based search. arXiv preprint arXiv:1511.09422 (2015)
H. Ding, D.A. Castañón, Optimal solutions for adaptive search problems with entropy objectives. arXiv preprint arXiv:1508.04127 (2015)
A.M. Axelrod, S.A. Karaman, G.V. Chowdhary, Exploitation by informed exploration between isolated operatives for information-theoretic data harvesting, in Conference on Decision and Control (CDC), Osaka, vol. 54, 2015
A. Axelrod, G. Chowdhary, Uninformed-to-informed exploration in unstructured real-world environments, in Self-Confidence In Autonomous Systems, Washington, DC (AAAI, 2015)
A. Axelrod, G. Chowdhary, A hybridized bayesian parametric-nonparametric approach to the pure exploration problem, in Bayesian Nonparametrics: The Next Generation, Montreal. Neural Information Processing Systems, 2015
A. Axelrod, G. Chowdhary, The Explore-Exploit Dilemma in Nonstationary Decision Making under Uncertainty, 1st edn. (Springer, New York, 2015), pp. 2198–4182
P. Bodik, W. Hong, C. Guestrin, S. Madden, M. Paskin, R. Thibaux, Intel lab data. Technical report, Intel Berkely Research Lab, Feb 2004
P. Berrisford, D.P. Dee, K. Fielding, M. Fuentes, P. Kallberg, S. Kobayashi, S. Uppala, The era-interim archive (European Center for Medium Range Weather Forecasts, Reading, 2009)
J. Haslett, A.E. Raftery, Ireland wind data set. Technical report, Trinity College and University of Washington, pp. 1961–1978
M. Widmann, C.S. Bretherton, Validation of mesoscale precipitation in the NCEP reanalysis using a new gridcell dataset for the northwestern united states. J. Clim. 13(11), 1936–1950 (2000)
Clean Air Status and Trends Network (CASTNET), Accessed Hourly Ozone data at www.epa.gov/castnet on Sept 2015
D.P. Dee, S.M. Uppala, A.J. Simmons, P. Berrisford, P. Poli, S. Kobayashi, U. Andrae, M.A. Balmaseda, G. Balsamo, P. Bauer et al., The era-interim reanalysis: configuration and performance of the data assimilation system. Q. J. R. Meteorol. Soc. 137(656), 553–597 (2011)
A. Gelman, J.B. Carlin, H.S. Stern, D.B. Dunson, A. Vehtari, D.B. Rubin, Bayesian data analysis, 3rd edn. (CRC Press, Boca Raton, 2013)
J. Duchi, Derivations for Linear Algebra and Optimization (Berkeley, 2007)
M.J Schervish, Theory of Statistics (Springer, New York, 2012)
Acknowledgements
This work is sponsored by the Air Force Office of Scientific Research Award Number FA9550-14-1-0399, and the Air Force Office of Scientific Research Young Investigator Program Number FA9550-15-1-0146.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Axelrod, A., Carlone, L., Chowdhary, G., Karaman, S. (2018). Data-Driven Prediction of Confidence for EVAR in Time-Varying Datasets. In: Blasch, E., Ravela, S., Aved, A. (eds) Handbook of Dynamic Data Driven Applications Systems. Springer, Cham. https://doi.org/10.1007/978-3-319-95504-9_16
Download citation
DOI: https://doi.org/10.1007/978-3-319-95504-9_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-95503-2
Online ISBN: 978-3-319-95504-9
eBook Packages: Computer ScienceComputer Science (R0)