Skip to main content
SpringerLink
Log in

Universal Hidden Monotonic Trend Estimation with Contrastive Learning

  • Conference paper
  • First Online:
Intelligent Computing (SAI 2023)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 739))

Included in the following conference series:

  • 550 Accesses

Abstract

In this paper, we describe a universal method for extracting the underlying monotonic trend factor from time series data. We propose an approach related to the Mann-Kendall test, a standard monotonic trend detection method and call it contrastive trend estimation (CTE). We show that the CTE method identifies any hidden trend underlying temporal data while avoiding the standard assumptions used for monotonic trend identification. In particular, CTE can take any type of temporal data (vector, images, graphs, time series, etc.) as input. We finally illustrate the interest of our CTE method through several experiments on different types of data and problems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 219.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 279.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Pineau, E., Razakarivony, S., Bonald, T.: Unsupervised ageing detection of mechanical systems on a causality graph. In: ICMLA (2020)

    Google Scholar 

  2. Jr Miller, R.G.: Survival Analysis. John Wiley & Sons, Hoboken (2011)

    Google Scholar 

  3. Huang, S.H., Mahmud, K., Chen, C.J.: Meaningful trend in climate time series: a discussion based on linear and smoothing techniques for drought analysis in Taiwan. Atmosphere 13(3), 444 (2022)

    Article  Google Scholar 

  4. Harvey, A.C., Shephard, N.: 10 structural time series models (1993)

    Google Scholar 

  5. Choi, S., Cichocki, A., Park, H.-M., Lee, S.-Y.: Blind source separation and independent component analysis: a review. Neural Inf. Proc.-Lett. Rev. 6(1), 1–57 (2005)

    Google Scholar 

  6. Hyvärinen, A., Oja, E.: Independent component analysis: algorithms and applications. Neural Netw. 13(4–5), 411–430 (2000)

    Article  Google Scholar 

  7. Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)

    Article  Google Scholar 

  8. Hyvarinen, A., Morioka, H.: Unsupervised feature extraction by time-contrastive learning and nonlinear ICA. In: Advances in Neural Information Processing Systems,, pp. 3765–3773 (2016)

    Google Scholar 

  9. Chen, K., Wang, J.: Design of multivariate alarm systems based on online calculation of variational directions. Chem. Eng. Res. Des. 122, 11–21 (2017)

    Article  Google Scholar 

  10. Niknam, S.A., Kobza, J., Hines, J.W.: Techniques of trend analysis in degradation-based prognostics. Int. J. Adv. Manuf. Technol. 88(9–12), 2429–2441 (2017)

    Article  Google Scholar 

  11. Le-Khac, P.H., Healy, G., Smeaton, A.F.: Contrastive representation learning: a framework and review. IEEE Access 8, 193907–193934 (2020)

    Article  Google Scholar 

  12. Franceschi, J.Y., Dieuleveut, A., Jaggi, M.: Unsupervised scalable representation learning for multivariate time series. In: Advances in Neural Information Processing Systems, pp. 4650–4661 (2019)

    Google Scholar 

  13. Banville, H., Albuquerque, I., Hyvarinen, A., Moffat, G., Engemann, D.A., Gramfort, A.: Self-supervised representation learning from electroencephalography signals. In: IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP), pp. 1–6. IEEE 2019 (2019)

    Google Scholar 

  14. Wang, X., Gupta, A.: Unsupervised learning of visual representations using videos. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2794–2802 (2015)

    Google Scholar 

  15. Mann, H.B.: Nonparametric tests against trend. Econometrica 13(3), 245–259 (1945)

    Article  MathSciNet  MATH  Google Scholar 

  16. Goh, C.: Econ 2 0A: sufficiency, minimal sufficiency and the exponential family of distributions (2001)

    Google Scholar 

  17. Thomas, O., Dutta, R., Corander, J., Kaski, S., Gutmann, M.U.: Likelihood-free inference by ratio estimation. arXiv preprint: arXiv:1611.10242 (2016)

  18. Gutmann, M.U., Dutta, R., Kaski, S., Corander, J.: Likelihood-free inference via classification. Stat. Comput. 28(2), 411–425 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  19. Goldsmith, F.B.: Monitoring for Conservation and Ecology, vol. 3. Springer Science & Business Media, Cham (2012)

    Google Scholar 

  20. Kendall, M.G.: In: Griffin, C (ed.) Rank Correlation Methods. 4th ed. (1975)

    Google Scholar 

  21. Gilbert, R.O.: Statistical Methods for Environmental Pollution Monitoring. Wiley, Hoboken (1987)

    Google Scholar 

  22. Gray, K.L.: Comparison of trend detection methods (2007)

    Google Scholar 

  23. Huang, N.E., et al.: The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proc. R. Soc. London. Ser. A: Math., Phys. Eng. Sci. 454(1971), 903–995 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  24. Laura, A.-M., José, M.-F., María, P., Ángel, G.-Z., Pilar, B.-V., Jaime, G.: Analysis of soil moisture trends in Europe using rank-based and empirical decomposition approaches. Global Planet. Change 215, 103868 (2022)

    Article  Google Scholar 

  25. Carmona, A.M., Poveda, G.: Detection of long-term trends in monthly hydro-climatic series of Colombia through Empirical Mode Decomposition. Clim. Change 123(2), 301–313 (2014)

    Article  Google Scholar 

  26. Wei, F., et al.: Vegetation dynamic trends and the main drivers detected using the ensemble empirical mode decomposition method in east Africa. Land Degrad. Dev. 29(8), 2542–2553 (2018)

    Article  Google Scholar 

  27. Zhang, J., et al.: Serial-EMD: fast empirical mode decomposition method for multi-dimensional signals based on serialization. Inf. Sci. 581, 215–232 (2021)

    Article  MathSciNet  Google Scholar 

  28. Saxena, A., Goebel, K.: Turbofan engine degradation simulation data set. NASA Ames Prognostics Data Repository (2008)

    Google Scholar 

  29. Adamowski, K., Prokoph, A., Adamowski, J.: Development of a new method of wavelet aided trend detection and estimation. Hydrol. Proc.: Int. J. 23(18), 2686–2696 (2009)

    Article  Google Scholar 

  30. Hyvarinen, A., Sasaki, H., Turner, R.: Nonlinear ICA using auxiliary variables and generalized contrastive learning. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 859–868. PMLR (2019)

    Google Scholar 

  31. Wiskott, L., Sejnowski, T.J.: Slow feature analysis: unsupervised learning of invariances. Neural Comput. 14(4), 715–770 (2002)

    Article  MATH  Google Scholar 

  32. Blaschke, T., Zito, T., Wiskott, L.: Independent slow feature analysis and nonlinear blind source separation. Neural Comput. 19(4), 994–1021 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  33. Schuler, M., Hlynsson, H.D., Wiskott, L.: Gradient-based training of slow feature analysis by differentiable approximate whitening. In: Asian Conference on Machine Learning, pp. 316–331. PMLR (2019)

    Google Scholar 

  34. Pineau, E., Razakarivony, S., Bonald, T.: Time series source separation with slow flows. In: ICML Workshop on Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models (2020)

    Google Scholar 

  35. Harrell, F.E., Jr., Lee, K.L., Mark, D.B.: Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat. Med. 15(4), 361–387 (1996)

    Article  Google Scholar 

  36. Steck, H., Krishnapuram, B., Dehing-Oberije, C., Lambin, P., Raykar, V.C.: On ranking in survival analysis: bounds on the concordance index. In: Advances in Neural Information Processing Systems, pp. 1209–1216 (2008)

    Google Scholar 

  37. Cox, D.R.: Regression models and life-tables. J. Roy. Stat. Soc.: Ser. B (Methodological) 34(2), 187–202 (1972)

    MathSciNet  MATH  Google Scholar 

  38. Katzman, J.L., Shaham, U., Cloninger, A., Bates, J., Jiang, T., Kluger, Y.: Deepsurv: personalized treatment recommender system using a cox proportional hazards deep neural network. BMC Med. Res. Methodol. 18(1), 24 (2018)

    Article  Google Scholar 

  39. Jing, B., et al.: A deep survival analysis method based on ranking. Artif. Intell. Med. 98, 1–9 (2019)

    Article  Google Scholar 

  40. Yu, C.N., Greiner, R., Lin, H.C., Baracos, V.: Learning patient-specific cancer survival distributions as a sequence of dependent regressors. In: Advances in Neural Information Processing Systems, pp. 1845–1853 (2011)

    Google Scholar 

  41. Bennett, S.: Analysis of survival data by the proportional odds model. Stat. Med. 2(2), 273–277 (1983)

    Article  Google Scholar 

  42. Fotso, S., et al.: PySurvival: open source package for survival analysis modeling (2019). https://www.pysurvival.io/

  43. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint: arXiv:1412.6980 (2014)

  44. Schumacher, M., et al.: Randomized 2 x 2 trial evaluating hormonal treatment and the duration of chemotherapy in node-positive breast cancer patients. German breast cancer study group. J. Clin. Oncol. 12(10), 2086–2093 (1994)

    Article  Google Scholar 

  45. Ishwaran, H., Kogalur, U.B., Blackstone, E.H., Lauer, M.S., et al.: Random survival forests. Ann. Appl. Stat. 2(3), 841–860 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  46. Fujisawa, H., Eguchi, S.: Robust parameter estimation with a small bias against heavy contamination. J. Multivar. Anal. 99(9), 2053–2081 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  47. Ghosh, A., Kumar, H., Sastry, P.: Robust loss functions under label noise for deep neural networks. arXiv preprint: arXiv:1712.09482 (2017)

  48. Han, B., et al.: A survey of label-noise representation learning: past, present and future. arXiv preprint: arXiv:2011.04406 (2020)

  49. Sasaki, H., Takenouchi, T., Monti, R., Hyvarinen, A.: Robust contrastive learning and nonlinear ICA in the presence of outliers. In: Conference on Uncertainty in Artificial Intelligence, pp. 659–668. PMLR (2020)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. EthiFinance, Paris, France

    Edouard Pineau, Mauricio Gonzalez & Anthony Schrapffer

  2. Safran Tech, Châteaufort, France

    Sébastien Razakarivony

Authors
  1. Edouard Pineau
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sébastien Razakarivony
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mauricio Gonzalez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anthony Schrapffer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Edouard Pineau .

Editor information

Editors and Affiliations

  1. Faculty of Science and Engineering, Saga University, Saga, Japan

    Kohei Arai

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pineau, E., Razakarivony, S., Gonzalez, M., Schrapffer, A. (2023). Universal Hidden Monotonic Trend Estimation with Contrastive Learning. In: Arai, K. (eds) Intelligent Computing. SAI 2023. Lecture Notes in Networks and Systems, vol 739. Springer, Cham. https://doi.org/10.1007/978-3-031-37963-5_36

Download citation

Publish with us

Policies and ethics

Search

Navigation