1 Introduction

Digital immersion is moving into public spaces. Interactive screens and public displays are deployed in urban environments, malls, and shop windows. Inner city areas, airports, train stations and stadiums are experiencing a transformation from traditional to digital displays enabling new forms of multimedia presentation and new user experiences. The market sees digital signage as more beneficial compared to static signage because content that updates frequently can be digitally updated, saving the cost of printing [12]. Digital signage also has the ability to be interactive with imbedded touch screens, movement detection and image capture devices. Czerwinski et al. [4] showed that larger displays improved recognition memory and peripheral awareness. This finding implies that people can more easily be made aware of instant information in public spaces as well, which will lead to effective provision of various content, such as advertisements, promotions, notices and so on. Tan et al. [13] reported a series of studies demonstrating the advantages of large displays on 3D navigation in virtual worlds. It is certain that large-scale display, in providing a wide field of view, helps users understand complex context in the view more easily. Furthermore, even if the visual angle is maintained, simply having a physically larger display improves performance on spatial tasks. Large displays naturally lend themselves to collaboration research as well based on their size, cost, and privacy concerns. Various papers have dealt with the use of large displays for collaboration. There is both a clear trend toward larger displays and mounting evidence that they increase user productivity and aid user recognition memory. However, G. Robertson et al. [10] showed that numerous usability problems inhibit the potential for even greater user productivity. P. Peltonen et al. [9] investigated a large multi-touch display installed in a central location in Helsinki, Finland. They analyzed how public availability is achieved through social learning and negotiation, and how the display restructures the public space. They found that the multi-touch feature, gesture-based interaction, and the physical display size contributed differentially to these uses. Furthermore, hedonic aspects are increasingly considered an important factor in user acceptance of information systems, especially for activities with high self-fulfilling value for users. J. Novak and S. Schmidt [8] demonstrated a higher hedonic stimulation quality of a touch-based large-display cooperative travel consultancy workspace than that of a traditional advisory setting. Coupled with qualitative user feedback highlighting visual qualities and touch-based interaction, these findings suggest the importance of intrinsic hedonic stimulation qualities in large-display visual workspaces.

In line with the previous researches, we propose novel contents visualization method composed of aligned layers, which supports multi user manipulation of large number of digital contents with multi-touch interaction. Figure 1 shows the SpreadView operating on 246-inch multi-touch integrated large display. The SpreadView is directed to maximize the number of pieces of digital contents displayed on a restricted screen. This method is composed of five steps of process – generating a page corresponding to a content file included in a folder; generating a folder display portion corresponding to the folder and an information display layer including the page; outputting the information display layer on a content display apparatus; sensing touch of the content display apparatus; and changing an output format of the information display layer based on the sensed touch. The generating the information display layer step includes aligning the folder display portion and the page of the information display layer in one direction. Owing to the distinctive features described above the SpreadView provides fast and easy interaction for searching and indexing among various types of digital contents without loss of space on display. This interaction concept may give designers a variety of inspirations such as intuitive user interaction with a number of virtual objects in 3D space or new user interface design to point, designate and select an object among big amount of candidates on large size of space or surface.

Fig. 1.
figure 1

The SpreadView provides multiple contents display view controlled by user’s instant touch manipulation.

Full size image

2 Related Work

Interactive large displays are deployed increasingly in a variety of settings, including exhibitions, events, museums, and other public places [8]. As situated media technology is rapidly maturing, it is likely that the trend will accelerate, so that people will become more accustomed to the large interactive walls. There are various researches about use of public displays situated in indoor and public environment. Morales-Arnada and Mayora-Ibarra introduced Dymo [11], a situated large interactive display outside of the workplace, within shared and sociable spaces such as common areas at universities and conferences, cafes, and hotel foyers designed to enable the sharing and exchange of a wide variety of digital media. Similarly, Fass et al. designed MessyBoard [6], an large, projected, shared bulletin board that is decorated collaboratively by a small group of users. Santa and Barra[16] proposed semi-public displays as a universal way for publicizing internal information and collaborative working in an office environment without other traditional means. Hinrichs and Carpendale [7] described findings from a field study that was conducted at the Vancouver Aquarium to investigate how visitors interact with a large table exhibit using multi-touch gestures. Visualizing contents of public display is one of the main research topics in this area. EMDialog [8] proposed by Hinrichs et al. provides a visual exploration environment for an artist’s work in a museum that offers interplay between two integrated visualizations along temporal and contextual dimensions. Similarly, Bohemian Bookshelf [19] aims to support serendipitous discoveries in the context of digital book collections which can be facilitated through visualization. Clases and Moere [3] present Street Infographics, an urban intervention that visually represent data that is contextually related to local issues, and is visualized through situated displays that are placed within the social and public context of an urban environment.

Interaction for browsing is also a matter of concern to many researchers in designing interactive public display field. Coutrix et al. [4] suggested FIZZYVis, a walk-up-and-use interface that displays information through bubbles reacting to touches, and its design goals. Peltonen et al. [14] examined interaction with the system, CityWall, a large multi-touch display installed in public space in various social configuration. Alt et al. present a digital public notice area Digifieds [1] which is built to understand emerging practices and provide easy and straight forward interaction techniques to be used for creating and exchanging content. They demonstrate that some challenges have to be confronted when designing interactive public displays: attention needs to be attracted; interactivity needs to be communicated [12]; the user need to be motivated to interact [2, 9]; suitable interaction techniques need to be provide. This work focuses on second and fourth challenges.

3 SpreadView: Design

3.1 System Structure

The SpreadView is composed of three main devices and two databases. The Fig. 2 is a diagram showing an internal configuration of the SpreadView for displaying content. The system includes a touch sensor, a conversion controller, a display unit, a file database and a conversion information DB.

Fig. 2.
figure 2

An internal configuration of the SpreadView

Full size image

The contents are multimedia contents including image, video, animated picture and so on. All or part of content can be displayed according to content types. For example, if content is an image the entire image or a part of the image can be displayed on the screen. If content is a video, a captured scene of a representative screen may be used as a part of content displayed on the content display apparatus and the reproduced video may be displayed on the content display apparatus. Content is displayed in correspondence with actual content and an object corresponding to (selected or extracted from) content is referred to as a “page”. The touch sensor detects or senses touch of the display. The touch sensor receives user input based on haptic or tactile touch. The touch sensor includes various software components for performing various operations such as an operation for determining whether the content display apparatus is touched, an operation for determining whether a touch point is moved and an operation for tracking movement of a touch point. The movement of a touch point is divided into one-point touch and two-point touch. One-point touch is defined as instantaneous touch if a touch property is less than a predetermined touch property threshold and is defined as continuous touch if a touch property is greater than the threshold value. The conversion controller serves to change an output format of an information display layer based on touch information sensed by the touch sensor. The display unit serves to output a folder display portion corresponding to a folder and an information display layer including a page corresponding to a content file in a folder. That is, the display unit visually displays the page to the user. The display unit combines with the touch sensor. The file DB serves to store content, which DB stores content in a tree structure of folders. That is, since files are stored in a hierarchical structure, content of a single folder may be displayed and higher folders of content files may be represented in an information display layer. The conversion information DB serves to store output information changed by the conversion controller. The conversion controller includes a one-point touch controller and a two-point touch controller. The one-point touch controller changes the output format of the display unit if one point is touched. The two-point touch controller changes the output format of the display unit if two points are touched. The two-point touch is applied to only the case where two points are included in the same information display layer. If two points are respectively included in different information display layers, each of the two points may be perceived as one point touched in each information display layer. In case of two-point touch, a determination as to whether two points are touched in a single page is first made. If two points are touched in the single page, the screen output format is changed depending on whether the touched two points are moved at the same velocity or at different velocities. If the two points are not touched in the single page, that is, if the two points are touched in different pages, a screen output format is separately changed.

3.2 Operation Process

The Fig. 3 is a flowchart illustrating the operation process. The system loads and checks file data from the file DB, where the folder information and the file information stored in the file DB is checked. The system generates a folder display portion corresponding to a folder and generates a page corresponding to a file using all or part of the loaded file information. The folder display portion and the page of the information display layer may be aligned in one direction, that is, in a horizontal direction or a vertical direction. And system loads the existing conversion information stored in the conversion information DB and outputs an information display layer including the folder display portion and the page on the screen. If the existing conversion information is not present, initialization is performed, that is, the information display layer is equally or randomly divided so as to output the folder display portion and the page. The system always waits for a touch operation, maintains the existing screen if a touch operation is not performed, and determines a screen display method of the page according to touch velocity.

Fig. 3.
figure 3

The operation process of the SpreadView

Full size image

If a one-point touch operation is performed, it is determined whether or not touch is instantaneous touch. If touch is instantaneous touch, one page is maximally enlarged and displayed on the entire screen and, if touch is not instantaneous touch, that is, if touch is continuous touch, the size of the page adjacent to the page corresponding to one point is changed and the output position of the page corresponding to one point is moved. On the other hand, if the one-point touch operation is not performed, a two-point touch operation is performed. If two points included in a single page are touched, the screen display method of the page differs according to movement velocities of two touch points represented two-dimensionaliy. The velocity includes the size and direction of the movement. If the two points represented two-dimensionally are moved at different velocities, the touched page is enlarged based on a movement path. If the velocities of the two points represented two-dimensionally are the same, the page including the touch points are moved along the information display layer. If the two touch points are moved at the same velocity in a direction perpendicular to the information display layer direction, the display position of the layer is changed. If two points included in different pages are touched, the sizes of the pages are equally divided and displayed on the screen.

3.3 Interaction

Figure 4(a) is a diagram showing an output screen of displaying multiple contents such as an image, a video, digital document by dividing the screen of the display area in a horizontal direction. The information display layer may be referred to as a page set including a plurality of pages. Figure 4(b) and (c) show three information display layers. A plurality of pages included in a folder based on an internal tree structure of a file system is represented on one information display layer. If another folder among folders represented in the folder display portion is selected, the page of the selected folder may be represented on a content display unit. Since all the internal angles between the edges of a rectangle are 90°, if the positions of two diagonally facing points are defined, a unique rectangle may be defined, the screen output method can be described using coordinates of a left upper end and coordinates of a right lower end of a page.

Fig. 4.
figure 4

A layout pattern of the multiple digital contents on the SpreadView

Full size image

One-Point Instantaneous Touch Interaction. In case of one-point instantaneous touch, as shown in Fig. 5, it is assumed that left upper end coordinates of the page before touch are (x1, y1) and right lower end coordinates of the page before touch are (x2, y2) and that left upper end coordinates of the page after touch are (x1′, y1′) and right lower end coordinates of the page after touch are (x2′, y2′). The touch page is output on the entire screen based on the maximum width and length of the display. It is possible to prevent the page from being cut by adjusting the length of the page to the length of the screen. Thus, the coordinates of the page may be represented according to the length of the page.

Fig. 5.
figure 5

The page is enlarged after one-point instantaneous touch

Full size image

One-Point Continuous Touch Interaction. Figure 6 is a diagram showing a screen output method upon one-point continuous touch. If a middle page “4” is touched and the touch point is continuously moved to the right, the width of a left page “3” is increased without changing the size of the page “4”. If the touch point of the page “4” is further moved to the right, a right page “5” is overlapped and is not displayed meaning that the page “4” is folded.

Fig. 6.
figure 6

The adjacent pages are changing their width according to the one-point continuous touch on the page.

Full size image

Two-Points Continuous Touch Interaction. If two touch points are moved at different velocities in one page, a screen is output while the page is enlarged or reduced and the left and right pages and the upper and lower information layers are increased or decreased. As shown in Fig. 7(a), two touch points are selected in one page, which are moved by different sizes in different directions. The size of the page, the sizes of the left and right pages, and the sizes of the upper and lower information display layers are changed according to the movement of the touch points.

Fig. 7.
figure 7

Two-points continuous touch interaction

Full size image

In case of two points in the same direction, as shown in Fig. 7(b) and (c), if the two touch points are moved along the information display layer at the same velocity, all the pages of the corresponding information layer may be moved regardless of the change in size of individual page. For example, if the two touch points are moved in the vertical direction of the information display layer at the same velocity, the corresponding information layer may be vertically moved regardless of change in size of the individual page.

Two-Points of Individual Pages Interaction. Figure 8 shows a screen output method upon touch of two points of individual pages. If two points of different pages are touched, the sizes of the pages interposed between both pages are equally divided and output on the screen.

Fig. 8.
figure 8

Two-points interaction of individual pages; the sizes of the pages interposed between both pages are equally divided

Full size image

4 Implementation

We developed 246 inch interactive large display which consists of an multi-touch sensor array comprised of LED components (emitters and receivers), a signal controller, PC interface software, and an application driver program. ARM 7 was used for detecting sensor signals and serializing and interfacing a LAN connection between the sensor frame and the computer. The test software for the prototype was developed using Visual C in the Win XP environment.

4.1 Hardware

Figure 9 shows the overall construction of our multi-touch system. The frame possesses a PCB, Printed Circuit Board and IR arrays of transmitters and receivers which are mounted around the frame. Each LED modules consist of a transmitter and receiver pair. When the left side of the array is activated as a transmitter, MCU switches the right side of the array as a receiver to detect IR light from the left side. Light detection information is transferred into MCU, which analyzes and converts the information into a data table where a 1 indicates that light was successfully received and a 0 indicates detection failure. The size of the data table depends on the number of LED modules. The data table is converted to serialized TCP/IP protocol and LAN Port transfers the data to PC via LAN RJ45.

Fig. 9.
figure 9

Overall construction of our multi-touch system.

Full size image

4.2 Software

To verify the data from hardware, raw data monitoring software is necessary. The left side of Fig. 10 shows the screen shot of raw data monitor program. The raw data from the data table is represented as shown where a red mark in the matrix means successful detection of light. By using this program, we could monitor how large the transmitter emission angle is and whether every IR module is operating normally or not. This software not only provides the sensing condition of the system but also provides its operating speed by indicating the elapsed time of scanning cycles. This program was developed with Visual C on Windows XP.

Fig. 10.
figure 10

Multi-touch detection to generate touch information for application programs.

Full size image

5 Conclusion

We have presented a design for displaying digital content on interactive large display. The method for displaying content includes: generating a page corresponding to a content file included in a folder; generating a folder display portion corresponding to the folder and an information display layer including the page; outputting the information display layer on a content display apparatus; sensing touch of the content display apparatus; and changing an output format of the information display layer based on the sensed touch. The key feature of the SpreadView is ‘space effectiveness’ that is ‘no waste of space between pages’ regardless of page layout pattern and ‘intuitive and easy to control’ such as zoom, move, alinement and reconfiguration through simple touch gestures. The information display layer includes aligning the folder display portion and the page layers in one direction so that user scroll each layers parallel to others without deterioration of page alignment. And SpreadView provides effective control interaction for rearrangement of size of layers and pages. To change the size of the information display layers, user touches one point and drag while touching and holding another layer, which makes the size of layer sets rearranged equally. With same manner, when user touches two pages which belongs to same layer, the SpreadView equally resized the pages placed among two selections. These unique manipulation methods are the special design for the large display composed of large numbers of information, which has not been applied in conventional window based GUI.

We believe that designers will be well served in considering the issues we have presented and applying the tools we have provided in this study. The provision of effective contents visualization with intuitive manipulation could lead to in-depth engagement in display content, which may successfully deliver such information as ads, notices, campaigns, and so on. And we expect the SpreadView would elevate the recognition degree of digital contents arranged on 2D or 3D information space, which leads users to in-depth understanding to the contents, providing a more attractive experience, which results in effective transfer of information with intuitive operation of large number of digital contents, which is beneficial not only to public large display but also to other computing environments such as tabletop computing, virtual reality and augmented reality display etc.