Skip to main content
SpringerLink
Log in

Photometric redshift estimation based on data mining with PhotoRApToR

  • Original Article
  • Published:
Experimental Astronomy Aims and scope Submit manuscript

Abstract

Photometric redshifts (photo-z) are crucial to the scientific exploitation of modern panchromatic digital surveys. In this paper we present PhotoRApToR (Photometric Research Application To Redshift): a Java/C ++ based desktop application capable to solve non-linear regression and multi-variate classification problems, in particular specialized for photo-z estimation. It embeds a machine learning algorithm, namely a multi-layer neural network trained by the Quasi Newton learning rule, and special tools dedicated to pre- and post-processing data. PhotoRApToR has been successfully tested on several scientific cases. The application is available for free download from the DAME Program web site.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. http://www.astro-wise.org/projects/KIDS/

  2. http://dame.dsf.unina.it/dame_photoz.html#photoraptor

  3. http://www.astro-wise.org/projects/KIDS/

  4. http://www.ivoa.net/documents/VOTable/

References

  1. Albrecht, A., Bernstein, G., Cahn, R., et al.: Report of the Dark Energy Task Force (2006)

  2. ANSI (American National Standards Institute), et al.: American National Standard Code for Information Interchange. The Institute (1977)

  3. Bengio, Y., LeCun, J.: In Large-Scale Kernel Machines. MIT Press (2007)

  4. Biviano, A., et al.: CLASH-VLT: The mass, velocity-anisotropy, and pseudo-phase-space density profiles of the z=0.44 cluster, galaxy MACS 1206.2-0847. A&A 558, A1 (2013). 22 pp

    Article  ADS  Google Scholar 

  5. Breiman, L.: Random Forests. Machine Learning, Springer Eds., 45, 1, 25–32 (2001)

  6. Brescia, M.: New Trends in E-Science: Machine Learning and Knowledge Discovery in Databases, 78 pages, Horizons in Computer Science Research. In: Clary, T.S. (ed.) Series Horizons in Computer Science. ISBN: 978-1-61942-774-7, Vol. 7. Nova Science Publishers (2012)

  7. Brescia, M., Cavuoti, S., Paolillo, M., Longo, G., Puzia, T.: The detection of Globular Clusters in galaxies as a data mining problem. MNRAS 421(2), 1155 (2012)

    Article  ADS  Google Scholar 

  8. Brescia, M., Cavuoti, S., D’Abrusco, R., Longo, G., Mercurio, A.: Photometric redshifts for Quasars in multi band Surveys. ApJ 772, 140 (2013)

    Article  ADS  Google Scholar 

  9. Brescia, M., et al.: DAMEWARE: A web cyberinfrastructure for astrophysical data mining. PASP 126(942), 783–797 (2014)

    ADS  Google Scholar 

  10. Brescia, M., Cavuoti, S., De Stefano, V., Longo, G.: A catalogue of photometric redshifts for the SDSS-DR9 galaxies. A&A 568, A126 (2014)

    Article  ADS  Google Scholar 

  11. Brescia, M., Cavuoti, S., Longo, G.: Automated physical classification in the SDSS DR10. A catalogue of candidate Quasars, MNRAS, accepted (in press) (2015)

  12. Byrd, R.H, Nocedal, J., Schnabel, R.B.: Math. Program. 63, 129 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  13. Capozzi, D., De Filippis, E., Paolillo, M., D’Abrusco, R., Longo, G.: The properties of the heterogeneous Shakhbazyan groups of galaxies in the SDSS. Mon. Not. R. Astron. Soc. 396(2), 900–917 (2009)

    Article  ADS  Google Scholar 

  14. Cavuoti, S., Brescia, M., Longo, G., Mercurio, A.: Photometric redshifts with Quasi Newton Algorithm (MLPQNA). Results in the PHAT1 contest. A&A 546(A13), 1–8 (2012)

    Google Scholar 

  15. Cavuoti, S., Brescia, M., D’Abrusco, R., Longo, G., Paolillo, M.: Photometric classification of emission line galaxies with machine-learning methods. MNRAS 437(1), 968–975 (2014)

    Article  ADS  Google Scholar 

  16. Collister, A.A., Lahav, O.: ANNz: Estimating Photometric Redshifts Using Artificial Neural Networks. PASP 116, 345 (2004)

    Article  ADS  Google Scholar 

  17. Connolly, A.J., Csabai, I., Szalay, A.S., Koo, D.C., Kron, R.G., Munn, J.A.: Slicing Through Multicolor Space: Galaxy Redshifts from Broadband Photometry. Astron. J. 110, 2655 (1995)

    Article  ADS  Google Scholar 

  18. Cybenko, G.: Approximations by superpositions of sigmoidal functions. Math. Control Signals Syst. 2, 303 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  19. The Dark Energy Survey Collaboration, The Dark Energy Survey, White Paper submitted to the Dark Energy Task Force, 42 pages. arXiv:0510346 (2005)

  20. Davidon, W.C.: SIAM Journal on Optimization (1991)

  21. Dietterich, T.: Overfitting and Undercomputing in Machine Learning. Comput. Surv. 27, 326 (1995)

    Article  Google Scholar 

  22. Drucker, H.: Improving regressors using boosting techniques. In: Proceedings of ICML97, pp. 107–115. Morgan Kaufmann Publishers Inc., San Francisco (1997)

  23. Euclid Red Book, ESA Technical Document, ESA/SRE(2011)12, Issue 1.1. arXiv:1110.3193 (2011)

  24. Farrow, D.J., et al.: Pan-STARRS1: Galaxy clustering in the Small Area Survey 2. MNRAS 437, 748–770 (2014)

    Article  ADS  Google Scholar 

  25. Geisser, S.: The predictive sample reuse method with applications. J. Am. Stat. Assoc. 70(350), 320–328 (1975)

    Article  MATH  Google Scholar 

  26. Groetsch, C.V.: The Theory of Tikhonov Regularization for Fredholm Equations of the First Kind, Pitman, Boston (1984)

  27. Hildebrandt, H., Arnouts, S., Capak, P., Wolf, C., et al.: A&A 523, 31 (2010)

    Article  ADS  Google Scholar 

  28. Hoaglin, D.C., Mosteller, F., Tukey, J.W.: Understanding Robust and Exploratory Data Analysis. Wiley C, New York (1983)

    MATH  Google Scholar 

  29. Hoyle, B., Rau, M.M., Zitlau, R., Seitz, S., Weller, J.: Feature importance for machine learning redshifts applied to SDSS galaxies, Sumitted to MNRAS. arXiv:1410.4696 (2014)

  30. Ilbert, O., Capak, P., Salvato, M., et al.: Cosmos Photometric Redshifts with 30-bands for 2−d e g 2. Astrophys. J. 690, 1236 (2009)

    Article  ADS  Google Scholar 

  31. Ivezic, Z., et al.: (the LSST team), The LSST Science Book, v2.0 (2009)

  32. Kearns, M.: In: Touretzky, D.S., Mozer, M.C., Hasselmo, M.E. (eds.) : A Bound on the Error of Cross Validation Using the Approximation and Estimation Rates, with Consequences for Training-Test Split, Neural Information Processing 8, pp. 183–189. Morgan Kaufmann (1996)

  33. Laureijs, R., et al.: Euclid Definition Study Report, ESA/SRE(2011)12, Issue 1.1 (2011)

  34. Marlin, B.M.: Missing data problems in machine learning. Library and Archives, Canada (2008)

    Google Scholar 

  35. Mobasher, B., Capak, P., Scoville, et al.: Photometric Redshifts of Galaxies in COSMOS. Astrophys. J. Suppl. Ser. 172(1), 117–131 (2007)

    Article  ADS  Google Scholar 

  36. Nissen, S.: Implementation of a Fast Artificial Neural Network Library. Technical Report. Department of Computer Science University of Copenhagen (DIKU) (2003)

  37. Peacock, J.A., Schneider, P., Efstathiou, G., et al.: ESA-ESO Working Group on Fundamental Cosmology, ESA-ESO Working Group on Fundamental Cosmology. Tech. Rep. (2006)

  38. Pedregosa, F., et al.: Scikit-learn: Machine Learning in Python. JMLR 12, 2825–2830 (2011)

    MATH  MathSciNet  Google Scholar 

  39. Pennebaker, W.B., Mitchell, J.L.: JPEG still image data compression standard, (3rd ed.) (1993)

  40. Provost, F., Fawcett, T., Kohavi, R.: The Case Against Accuracy Estimation for Comparing Induction Algorithms. In: Kaufmann, M. (ed.) Proceedings of the 15th International Conference on Machine Learning, pp. 445–553 (1998)

  41. Repici, J.: How To: The Comma Separated Value (CSV) File Format. Creativyst Inc (2010)

  42. Rosenblatt, F.: Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms. Spartan Books, Washington (1961)

    Google Scholar 

  43. Rubinstein, R.Y., Kroese D.P.: The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation, and Machine Learning. Springer, New York (2004)

  44. Serjeant, S.: Up to 100,000 Reliable Strong Gravitational Lenses in Future Dark Energy Experiments. ApJ 793(1), L10 (2014)

    Article  ADS  Google Scholar 

  45. Stehman, S.V.: Selecting and interpreting measures of thematic classification accuracy. Remote Sens. Environ. 62(1), 77–89 (1997)

    Article  Google Scholar 

  46. Tagliaferri, R., Longo, G., Andreon, S., Capozziello, S., Donalek, C., Giordano, G.: Neural Networks and Photometric Redshifts, Neural Nets. Lect. Notes Comput. Sci 2859, 226–234 (2002)

    Article  ADS  Google Scholar 

  47. Taylor, M.B.: STILTS - A Package for Command-Line Processing of Tabular Data. Proceedings of the Astronomical Data Analysis Software and Systems XV ASP Conference Series 351, 666 (2006)

    ADS  Google Scholar 

  48. Umetsu, K., Medezinski, E., Nonino, M., et al.: CLASH: Mass Distribution in and around MACS J1206.2-0847 from a Full Cluster Lensing Analysis. ApJ 755(1), 56 (2012)

    Article  ADS  Google Scholar 

  49. Wells, D.C., Greisen, E.W., Harten, R.H.: FITS: a Flexible Image transport System. Astron. Astrophys. Supplement Series 44, 363 (1981)

    ADS  Google Scholar 

Download references

Acknowledgments

The authors wish to thank the anonymous referee for all very useful comments and suggestions. MB wishes to thank the financial support of the 7th European Framework Programme for Research Grant FP7-SPACE-2013-1, VIALACTEA - The Milky Way as a Star Formation Engine. The authors also wish to thank the financial support of Project F.A.R.O. III Tornata (University Federico II of Naples). GL acknowledges financial contribution through the PRIN-MIUR 2012 Cosmology with the Euclid Space Mission.

Author information

Authors and Affiliations

  1. INAF - Astronomical Observatory of Capodimonte, Napoli, Italy

    S. Cavuoti & M. Brescia

  2. Department of Physics, University Federico II of Naples, Naples, Italy

    V. De Stefano & G. Longo

Authors
  1. S. Cavuoti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Brescia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. V. De Stefano
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. G. Longo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Brescia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cavuoti, S., Brescia, M., De Stefano, V. et al. Photometric redshift estimation based on data mining with PhotoRApToR. Exp Astron 39, 45–71 (2015). https://doi.org/10.1007/s10686-015-9443-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10686-015-9443-4

Keywords

Search

Navigation