Abstract
Graphical user interfaces are usually first sketched out manually as hand drawing pictures and then must be realized by software developers to become prototypes or usable user interfaces. This motivates our proposal of a smart CASE tool that can understand hand drawing sketches of graphical user interfaces, including forms and their navigations, then automatically transform such draft designs into real user interfaces of a prototype or an application. By using the ideas of modeling and model-transformation in model driven engineering, the authors also propose a mechanism to generate graphical user interfaces as forms targeting different platforms. Experimental results show that our sketch recognition to understand hand drawing graphical user interfaces can achieve the accuracy of 97.86% and 95% in recognizing 7 common UI controls and arrows for navigation respectively. Our model transformation engine can generate user interfaces as forms for applications on 3 different platforms of mobile devices, including Windows Phone, Android, and iOS. This approach follows the trend to develop a new generation of smart CASE tools that can understand and interpret conceptual software design models into concrete software elements and components to assist the software development process in a natural way.
Chapter PDF
Similar content being viewed by others
References
Hardesty, L.: Picture-driven computing (January 2010), http://web.mit.edu/newsoffice/2010/screen-shots-0120.html
Lewis, J.P.: Fast Template Matching. In: Vision Interface 1995, pp. 120–123. Candian Image Processing and Pattern Recognition Society, Quebec City (1995)
Ouyang, W., Cham, W.K.: Fast algorithm for Walsh Hadamard transform on sliding windows. IEEE Transaction on Pattern Analysis and Machine Intelligence, 165–171 (2010)
Hofhauser, A., Steger, C., Navab, N.: Edge-Based Template Matching and Tracking for Perspectively Distorted Planar Objects. In: Bebis, G., et al. (eds.) ISVC 2008, Part I. LNCS, vol. 5358, pp. 35–44. Springer, Heidelberg (2008)
Lowe, D.G.: Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision (IJCV), 91–110 (2004)
Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: SURF: Speeded Up Robust Features. Computer Vision and Image Understanding (CVIU), 346–359 (2008)
Dalal, N., Triggs, B.: Histograms of Oriented Gradients for Human Detection. In: Conference on Computer Vision and Pattern Recognition, pp. 886–893 (2005)
Sun, Z., Liang, S.: Sketch retrieval and relevance feedback with biased SVM classification. Pattern Recognition Letters, 1733–1741 (2008)
Su, M.C., Hsio, T.H., Hsieh, Y.Z., Lin, S.C.: A neural-network-based sketch recognition system. In: International Symposium on Intelligent Signal Processing and Communications Systems (ISPACS), pp. 420–423 (2012)
Eitz, M., Hildebrand, K., Boubekeur, T., Alexa, M.: Sketch-Based Image Retrieval: Benchmark and Bag-of-Features Descriptors. Transactions on Visualization and Computer Graphics, 1624–1636 (2011)
Mourouzis, A., Leonidis, A., Foukarakis, M., Antona, M., Maglaveras, N.: A Novel Design Approach for Multi-device Adaptable User Interfaces: Concepts, Methods and Examples. In: Stephanidis, C. (ed.) Universal Access in HCI, Part I, HCII 2011. LNCS, vol. 6765, pp. 400–409. Springer, Heidelberg (2011)
Balagtas-Fernandez, F.T., Hussmann, H.: Model-Driven Development of Mobile Applications. In: International Conference on Automated Software Engineering (ASE 2008), pp. 509–512 (2008)
Kherraf, S., Lefebvre, E., Suryn, W.: Transformation From CIM to PIM Using Patterns and Archetypes. In: 19th Australian Conference on Software Engineering (ASWEC 2008), pp. 338–346 (2008)
Hammond, T.A.: LADDER: A Perceptually-based Language to Simplify Sketch Recognition User Interface Development, Massachusetts Institute of Technology, Doctor of Philosophy thesis (2007)
Diep, C.-K., Tran, Q.-N., Tran, M.-T.: Online model-driven IDE to design GUIs for cross-platform mobile applications. In: The 4th Symposium on Information and Communication Technology, pp. 294–300 (2013)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Nguyen, VT., Tran, MT., Duong, AD. (2014). Picture-Driven User Interface Development for Applications on Multi-platforms. In: Kurosu, M. (eds) Human-Computer Interaction. Theories, Methods, and Tools. HCI 2014. Lecture Notes in Computer Science, vol 8510. Springer, Cham. https://doi.org/10.1007/978-3-319-07233-3_33
Download citation
DOI: https://doi.org/10.1007/978-3-319-07233-3_33
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-07232-6
Online ISBN: 978-3-319-07233-3
eBook Packages: Computer ScienceComputer Science (R0)