An Application of Intelligent Data Analysis Techniques to a Large Software Engineering Dataset

  • James Cain
  • Steve Counsell
  • Stephen Swift
  • Allan Tucker
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5772)


Within the development of large software systems, there is significant value in being able to predict changes. If we can predict the likely changes that a system will undergo, then we can estimate likely developer effort and allocate resources appropriately. Within object oriented software development, these changes are often identified as refactorings. Very few studies have explored the prediction of refactorings on a wide-scale. Within this paper we aim to do just this, through applying intelligent data analysis techniques to a uniquely large and comprehensive software engineering time series dataset. Our analysis show extremely promising results, allowing us to predict the occurrence of future large changes.


Software Engineering Time Series Analysis Hidden Markov Models 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Advani, D., Hassoun, Y., Counsell, S.: Extracting Refactoring Trends from Open-source Software and a Possible Solution to the ‘Related Refactoring’ Conundrum. In: Proceedings of ACM Symposium on Applied Computing, Dijon, France (April 2006)Google Scholar
  2. 2.
    BSCKIT Browser Toolkits for Microsoft Visual C++, Microsoft support knowledge base article number Q153393,
  3. 3.
    Cain, J.: Debugging with the DIA SDK, Visual System Journal (April 2004),
  4. 4.
    Chen, G., Banerjee, N., Jaradat, S.A., Tanaka, T.S., Ko, M.S.H., Zhang, M.Q.: Evaluation and Comparison of Clustering Algorithms in Analyzing ES Cell Gene Expression Data. Statistica Sinica 12, 241–262 (2002)MathSciNetMATHGoogle Scholar
  5. 5.
    Demeyer, S., Ducasse, S., Nierstrasz, O.: Finding refactorings via change metrics. In: ACM Conference on Object Oriented Programming Systems Languages and Applications (OOPSLA), Minneapolis, USA, pp. 166–177 (2000)Google Scholar
  6. 6.
    Efron, B., Tibshirani, R.J.: An Introduction to the Bootstrap. Chapman & Hall, London (1993)CrossRefMATHGoogle Scholar
  7. 7.
    Fowler, M.: Refactoring: Improving the Design of Existing Code. Addison-Wesley, Reading (1999)MATHGoogle Scholar
  8. 8.
    Mancoridis, S., Mitchell, B.S., Rorres, C., Chen, Y., Gansner, E.R.: Using automatic clustering to produce high-level system organizations of source code. In: Proceedings of the 6th International Workshop on Program Comprehension (IWPC 1998), Ischia, Italy, pp. 45–52. IEEE Computer Society Press, Los Alamitos (1998)Google Scholar
  9. 9.
    Mens, T., Tourwe, T.: A Survey of Software Refactoring. IEEE Transactions on Software Engineering 30(2), 126–139 (2004)CrossRefGoogle Scholar
  10. 10.
    Mens, T., van Deursen, A.: Refactoring: Emerging Trends and Open Problems (2003),
  11. 11.
    Murphy, K.: Dynamic Bayesian Networks: Representation, Inference and Learning. PhD Thesis, UC Berkeley, Computer Science Division (July 2002)Google Scholar
  12. 12.
    Opdyke, W.: Refactoring Object-Oriented Frameworks. PhD thesis, University of Illinois at Urbana-Champaign (1992)Google Scholar
  13. 13.
    Pietrek, M.: Under the Hood. MSDN Magazine 17(3) (2002)Google Scholar
  14. 14.
    Rabiner, L.R.: A tutorial on HMM and selected applications in speech recognition. Proceedings of the IEEE 77(2), 257–286 (1989)CrossRefGoogle Scholar
  15. 15.
    Schreiber, S.: Undocumented Windows 2000 Secrets, A Programmer’s Cookbook. Addison-Wesley, Reading (2001)Google Scholar
  16. 16.
    Stevens, W., Myers, G., Constantine, L.: Structured design. IBM Systems Journal 13(2), 115–139 (1974)CrossRefGoogle Scholar
  17. 17.
    Zhao, L., Hayes, J.: Predicting Classes in Need of Refactoring: An Application of Static Metrics. In: Proceedings of 2nd International PROMISE Workshop, Philadelphia, US (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • James Cain
    • 1
  • Steve Counsell
    • 2
  • Stephen Swift
    • 2
  • Allan Tucker
    • 2
  1. 1.Quantel LimitedBerkshireUK
  2. 2.School of Information Systems, Computing and MathematicsBrunel UniversityUxbridgeUK

Personalised recommendations