Assessing Simulated Software Graphs Using Conditional Random Fields
In the field of software evolution, simulating the software development process is an important tool to understand the reasons why some projects fail, yet others prosper. For each simulation however, there is a need to have an assessment of the simulation results. We use Conditional Random Fields, specifically a variant form based on the Ising model from theoretical physics, to assess software graph quality. Our CRF-based assessment model works on so called Software Graphs, where each node of that graph represents a software entity of the software project. The edges are determined by immediate dependencies between the pieces of software underlying the involved nodes.
Because there is a lack of reference training data for our kind of evaluation, we engineered a special training paradigm that we call the Parsimonious Homogeneity Training. This training is not dependent on reference data. Instead of that it is designed to produce the following two effects. First, homogenizing the assessment of highly interconnected regions of the software graph, Second, leaving the assessment of these regions in relative independence from one another.
The results presented demonstrate, that our assessment approach works.
KeywordsSimulating software graphs Conditional random fields Parsimonious homogeneity training
The authors thank the SWZ Clausthal-Göttingen (https://www.simzentrum.de/en/) that partially funded our work (both the former projects “Simulation-based Quality Assurance for Software Systems” and “DeSim”, and the recent project “Agent-based simulation models in support of monitoring the quality of software projects”).
- 2.Dong, Z., et al.: CRF-based models of protein surfaces improve protein-protein interaction site predictions. BMC Bioinform. 15, 277 (2014). https://doi.org/10.1186/1471-2105-15-277
- 4.Honsel, D., Honsel, V., Welter, M., Waack, S., Grabowski, J.: Monitoring software quality by means of simulation methods. In: Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM 2016), Article 11, 6 pp. ACM, New York (2016). https://doi.org/10.1145/2961111.2962617
- 5.Honsel, V., Honsel, D., Herbold, S., Grabowski, J., Waack, S.: Mining software dependency networks for agent-based simulation of software evolution. In: ASE Workshop (2015)Google Scholar
- 6.Holland, P.W., Laskey, K.B., Leinhardt, S.: Stochastic blockmodels: first steps. Soc. Netw. 5(2), 109–137 (1983). ISSN: 0378–8733. https://doi.org/10.1016/0378-8733(83)90021-7. (http://www.sciencedirect.com/science/article/pii/0378873383900217)
- 7.Sha, F., Pereira, F.: Shallow parsing with conditional random fields. In: Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - Volume 1, (NAACL 2003), vol. 1, pp. 134–141. Association for Computational Linguistics, Stroudsburg (2003). https://doi.org/10.3115/1073445.1073473
- 8.He, X., Zemel, R.S., Carreira-Perpinan, M.A.: Multiscale conditional random fields for image labeling. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, vol. 2, pp. II-695–II-702 (2004). https://doi.org/10.1109/CVPR.2004.1315232
- 9.McCallum, A., Li, W.: Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons. In: Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003 - Volume 4 (CONLL 2003), vol. 4, pp. 188–191. Association for Computational Linguistics, Stroudsburg (2003). https://doi.org/10.3115/1119176.1119206
- 11.Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of 18th International Conference on Machine Learning, pp. 282–289 (2001). citeseer.ist.psu.edu/lafferty01conditional.html
- 12.Wang, S.B., Quattoni, A., Morency, L.P., Demirdjian, D., Darrell, T.: Hidden conditional random fields for gesture recognition. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), pp. 1521–1527 (2006). https://doi.org/10.1109/CVPR.2006.132