Development and Evaluation of a New Platform for Accelerating Cross-Domain Data Exchange and Cooperation
- 111 Downloads
The technologies for collecting and analyzing data are developing significantly, enabling us to gain new knowledge from various types of data. Recently, the considerably increasing expectation for cross-domain data exchange and cooperation has attracted attention from data markets. However, as we lack data-utilization knowledge, an accurate evaluation of data is difficult; a non-price indicator for deciding the exchange of data is needed. Thus, in this study, we propose a new online platform for stakeholders to communicate regarding data utilization. Our online platform, called Web Innovators Marketplace on Data Jackets (Web IMDJ), is built with reference to the process of IMDJ workshops. Web IMDJ is superior to the conventional paper-based IMDJ (hereafter, Table IMDJ) in terms of reducing the burden on conducting workshops. Notably, Table IMDJ and Web IMDJ have different communication media, and this may affect the data-utilization knowledge proposed in the workshop. Therefore, we conducted workshops on both platforms under a controlled experimental environment to compare the proposed data-utilization knowledge. Consequently, the knowledge proposed in Web IMDJ gained equal or higher ratings by third-party evaluators (those who did not join in the experimental workshop). By contrast, subjects themselves evaluated the knowledge proposed in Table IMDJ as superior to Web IMDJ. These results revealed that both workshops have advantages as data-utilization platforms. Furthermore, we derived the best practices to utilize data effectively from a detailed analysis of the data obtained from the experimental workshops.
KeywordsData Jacket Face-to-face versus online Market of data Online data marketplace for innovators
Situations and Issues Surrounding Data Utilization
Recently, terms such as Big Data, artificial intelligence (AI), Internet of Things (IoT), and cloud computing, have become popular in various fields. These terms sometimes, called buzzwords, express vague concepts regarding the technology of the future, and indeed indicate the development of technology [1, 2, 3, 4]. For example, regarding Big Data and IoT, sensors are improving and could provide timely and hi-fidelity data . Smart city projects utilize city resources more efficiently with data acquired from sensors and are, thus, advancing worldwide . AI involves various technologies including the improvement and versatility of data analysis methods, such as regression analysis , classification by trees  or vectors , probabilistic classifiers , and artificial neural networks . Cloud computing represents the improvement of technology and service forms relating to resource distribution . Additionally, the growth of computers’ calculation speeds and environments conducive to learning data analysis increase our expectation of gaining new knowledge through data utilization.
Thus, why do these terms describe only vague concepts? One of the biggest reasons could be that we cannot utilize data as per our expectations, despite the improvement of various technologies. The challenge in fully utilizing data surrounds the difficulty in exchanging data. Data holders, especially companies, cannot publish their data for free because data are their asset. Consequently, data users experience difficulty obtaining the kind of data a company owns and may, therefore, be unable to acquire the data they actually need. Moreover, when we exchange data via a purchase, it is still challenging to set a reasonable price because of the lack of knowledge surrounding data utilization [13, 14, 15].
A data market, a platform to buy/sell data, works as a method for accelerating data exchange. There are various online markets such as Every Sense1 , Japan Data Exchange Inc.,2 Azure Marketplace3 , CKAN4 , Qlik DataMarket5 , and Dawex6 . Additionally, some research groups have surveyed the existing data markets and classified data marketplaces based on various aspects [21, 22, 23]. In these platforms, data holders provide the outlines and prices of their data based on which data users decide whether to buy the data. These pieces of information are useful in that they make it possible to search for existing data and the company that owns them. In Japan, some experiments utilizing such platforms have been conducted to verify whether they perform as expected.,7Japanese only. Accessed June 2019.8
However, the issue of data pricing still remains. Liang et al.  classified the existing data pricing models according to various aspects: economical [24, 25], game theory [26, 27], and auction [28, 29] based. More recently, various types of improved data pricing and trading models have been proposed [30, 31, 32], showing the high expectation of data exchange. Nonetheless, as discussed in , the wide diversity of data prevents us from optimal trading.
Indeed, we do not need to price data to exchange data. Here, we focus on the communication of stakeholders and invent data-utilization scenarios that describe the purpose of data exchange. Additionally, when data are traded based on the data-utilization scenarios, the risk of business opportunity loss can be reduced by devising a contract. Therefore, the devised scenarios assist not only in exchanging data at a reasonable price, but also in facilitating the collaborations needed to achieve a given scenario. Furthermore, by accumulating such scenarios and their prices as data-utilization knowledge, predicting the outcome of data utilization and determining the price of data itself in the future can be made possible.
Communication Platform for Data Utilization
Several data-utilization projects with IMDJ are in progress in various domains. For example, in Tokyo Marunouchi. a demonstration experiment is being conducted since May 2018, aiming at creating a new town by utilizing data from multiple industries.9 Therein, several projects have begun based on data-utilization scenarios proposed by IMDJ . Additionally, Kyodo Printing Co., Ltd. is working on data-distribution support projects with IMDJ.10. Thus, IMDJ has contributed to the promotion of data utilization.
However, IMDJ has a major problem in that it involves a large operational burden (see “Comparison of the Whole Process of Table IMDJ and Web IMDJ” section for details) to prepare a workshop and save the proposed knowledge. This not only results in disregarding the introduction of IMDJ to projects but also complicates the continuous discussion of data utilization with IMDJ. Indeed, we often conduct a workshop only once in IMDJ-introduced projects. This prevents the acceleration of data utilization and the accumulation of data-utilization knowledge. Moreover, from the viewpoint of innovation, we cannot expect much impact from a single workshop. To address this limitation, we propose a new online platform to reduce the burden of workshops. Also, by comparing IMDJ with our platform, we show how our platform contributes to promoting data utilization.
The outline of this paper is as follows. “Related Studies” section details the origin and process of IMDJ. Furthermore, to compare the conventional paper-based IMDJ (hereafter, Table IMDJ) with Web IMDJ, we introduce studies that compare face-to-face and online communications. “Web IMDJ” section introduces Web IMDJ, our online platform for conducting IMDJ workshops with less burden. In “Comparison Experiment of Table and Web IMDJs” section, we describe an experimental workshop for discussing the influence of different communication media, i.e., face-to-face and online communications in Table IMDJ and Web IMDJ, respectively. Moreover, we propose the most effective IMDJ operation method for promoting data utilization. Finally, we conclude this paper in “Conclusion” section.
I. Preparation for the workshop;
II. Implementation of the workshop;
III. Preservation of knowledge proposed.
II-A. Present requirements;
II-B. Propose solution scenarios;
II-C. Evaluate the solution scenarios.
In step II-A, participants present requirements related to the contents of DJs and keywords on the map. As such, it becomes easier to conduct discussions based on the DJ when devising solutions. The position of the participants is not determined to allow them to present any requirements. Requirements are represented by yellow sticky notes and placed near the related DJ.
In step II-B, participants create solution scenarios that represent a series of actions to meet the presented requirements based on the DJs. Solutions, i.e., items which seem useful to achieve the solution, are represented by blue sticky notes with the DJ IDs. As the number of DJs on the map is between 25 and 30, other data are often necessary for devising solutions. Therefore, we describe not only the existing data but also the data assumed to be acquired in the future by red sticky notes, as additional DJs.
In step II-C, participants evaluate the proposed solution scenarios. The participants can evaluate any solution except for their own. Here, the monkey bill, a virtual bill valid only in the workshop, is used for evaluation. Ten monkey bills are distributed to each participant at the beginning of the IMDJ workshop, and the participants ‘pay bills’ to evaluate the solutions. The number of paid bills is represented by a small paper that is attached to the evaluated solution.
Color of sticky notes and written content
Color of papers
Data Jacket card (DJ card)
Data user’s requests or social issues (Requirement)
Solution scenarios for requirements (Solutions)
Additional Data Jackets
Number of paid monkey bills
Finally, in the knowledge preservation phase, we transcribed all the content described in the papers and saved them to the database. As the data-utilization knowledge proposed in the workshop has a hierarchical structure of requirement, solution, and DJ, the knowledge is stored in the RDF/XML format . The hierarchical structure represents the network of solutions created by a combination of different types of DJs and requirements satisfied by multiple solutions. RDF/XML is a syntax, defined by the W3C, to express a resource description framework (RDF) graph as an extensible markup language (XML) document . The knowledge saved here is used in the demonstration experiments, as introduced in “Communication Platform for Data Utilization” section, and it is utilized to improve accuracy when searching for DJs.
Face-to-Face Versus Online Communication
The comparison of face-to-face and online communications has been widely studied in the fields of education and medical care. Especially, in the education field, the educational gap is such a critical issue that online education has attracted attention as a solution. The high expectations for online education are well expressed by the Babson Survey Research Group, which has been publishing reports on American online education every year since 2003; it shows that the number of students receiving online education is increasing every year . This report also reveals that the percentage of academic leaders who rate the learning outcomes from online education similar or superior to that from face-to-face instruction is increasing: 71.4% and 57.2% in the surveys of 2016 and 2003, respectively. Thus, online education has not only been improving, but also has already achieved a high level. By contrast, Bowers et al.  pointed out that in online education, students scarcely communicate with other students and often quit learning along the way because of feelings of isolation. As a result, blended learning, which combines conventional face-to-face and online education systems, has attracted attention; various studies have stated that blended learning improves student performance . The aforementioned studies reveal that online education has been improved to a level equal to or higher than that face-to-face education; however, further promotion of blended education is necessary to improve student performance.
A similar trend exists in medical treatment. Olthuis et al.  compared the extent to which treatment is impacted by online cognitive behavior therapy (CBT) with therapist support, online CBT without any support, face-to-face CBT, and no treatment. They determined that online and face-to-face CBTs showed no significant differences in clinically meaningful improvement in anxiety. This study suggests that online CBT has improved to a level equal to that of face-to-face CBT. Wentzel et al.  argued that it is essential for patients and doctors to mutually understand treatment, especially in blended (online and face-to-face) mental health care; they proposed a framework that promotes mutual understanding, showing that the environment for blended mental care has been developing towards practical implementation.
Although the aforementioned studies focus on the one-to-one relationship between teachers and students/doctors and patients, some studies focused on team communication. Salter et al.  compared face-to-face and online communications in a reflective discussion that encouraged students to consider and talk about what they had observed. They found that face-to-face communication tends to cause divergent arguments, whereas online communication leads to exhaustive arguments. They also argued that threading arguments in face-to-face communication and continuing to discuss online communication promoted collective understanding. Christina et al.  focused on the relationship between team trust and team effectiveness and reported it to be stronger in virtual teams than in face-to-face teams. Thus, team trust is more critical in virtual teams; however, when dialogs from virtual team interactions were documented, the relationship weakened. They, thus, concluded that documenting team interactions is a viable complement to trust-building activities, particularly in virtual teams. Abrams et al.  compared data richness trade-offs between face-to-face and online focus groups; compared to face-to-face focus groups, online groups did not facilitate rich data.
Comparisons between face-to-face and online communications have also been studied in workshop methods derived from chance discovery. Some researchers have focused on the innovators marketplace (IM), a predecessor workshop method of IMDJ. They reported that both the feasibility and novelty of the business proposals put forward in online workshops received high evaluation . It should be noted that subjects were already in a relationship of trust in advance; this seemed to affect the result. Wang et al. [49, 50] built an online system called iChance based on IM and reported that online discussion invented more ideas than paper-based IM.
These various studies overviewed in this section illustrate some of the differences in communication media methods. Furthermore, the studies conducted on this topic in various fields show that the utility of online communication is recognized, and that the expectation for common implementation is high. Moreover, instead of the shifting trend toward online communication, blended communication, which can successfully combine the advantages of both face-to-face and online communications, has attracted attention. This indicates that face-to-face and online communications have their respective advantages; this can also be applied to the discussion on data utilization. Therefore, our mission is not only to promote data utilization by implementing Web IMDJ but also to ascertain the optimal operation method by understanding the advantages of both Table (face-to-face) and Web (online) IMDJ.
As discussed in “Situations and Issues Surrounding Data Utilization” section, Web IMDJ focuses on discussions surrounding data utilization among stakeholders. In other words, data exchange or collaborations can occur based not on the data price, but on the discussion surrounding data utilization. Additionally, as Web IMDJ is an online platform that reduces the burden of a workshop, data utilization can continually be easily discussed.
In this chapter, we introduce the detailed structure and implementation of Web IMDJ to show that we can conduct a Web IMDJ workshop in the same sense as a Table IMDJ workshop. Additionally, we compared the processes of Table and Web IMDJ to show that Web IMDJ reduces the burden of workshops.
Structure of Web IMDJ
Events defined in the workshop page, with the process that they trigger, and information stored in the database for that event
Participate in IMDJ
Show new participant’s username on the page
Participant’s username, ID
Click papers on the upper right area
Create text area with info as ID, proposer, etc.
Given attributes to the text area
Drag and drop the text area on map
Show the text area on every participant’s map
Rewrite text area
Reflect changes in all participant’s page
Update the content
Associate ideas and DJs
Reflect the association in case ideas are shared
Comment by chat
Reflect the comments
Evaluate with monkey bills
Reflect the transaction and attach the number of paid bills to the solution
Exit IMDJ page
Delete the participant’s username from the page
Implementation of Web IMDJ
However, Table and Web IMDJ have fundamental differences in face-to-face and online communication media; this may affect the number and quality of the proposed data-utilization knowledge. For example, if the number and quality of data-utilization knowledge proposed in Web IMDJ are inferior to those of Table IMDJ for one workshop, the effect of Web IMDJ becomes questionable. In other words, it is essential to guarantee the number and quality of data-utilization knowledge in Web IMDJ. “Comparison Experiment of Table and Web IMDJs” section presents a comparison of Table and Web IMDJ under a controlled situation with regard to the number and quality of the proposed solutions as well as in the knowledge-creation process.
Comparison of the Whole Process of Table IMDJ and Web IMDJ
As described, although it is essential to repeat the workshop to improve the scenarios, the workshop is often conducted only once per project because of the heavy burden of the workshop. In this chapter, we compare the processes of Table and Web IMDJ for each phase of the workshop to prove that Web IMDJ reduces this burden dramatically.
In Phase I (“IMDJ” section), we determined the workshop’s purpose, theme, and which DJs to use first. After determining this required information, we visualized the relationship of DJs as a network diagram. In Table IMDJ, the following six steps must be followed: (A) disassemble contents of DJ into sentence elements and output in comma-separated values (CSV) format; (B) import the CSV file into a visualization tool; (C) visualize a map using the KeyGraph algorithm ; (D) arrange the visualized map for smooth discussion; (E) print the visualized map on large-sized sheets (typically, size A0 or B0). In addition to these steps, DJ cards must be created.
On the other hand, in Web IMDJ, owing to automation, we only need to input DJ IDs and set several parameters to visualize a map. Additionally, as Web IMDJ allows for conducting online workshops, the map does not need to be printed. As a result, Web IMDJ reduces the time required for preparation by approximately 1–2 h.
In Phase II (“IMDJ” section), the process of the workshop is the same in both IMDJs; however, in Table IMDJ, we need to arrange the date and place. In contrast, Web IMDJ allows users to participate from remote locations if they have a computer connected to the network.
In Phase III (“IMDJ” section), all knowledge proposed in Table IMDJ must be converted into digital data; whereas in Web IMDJ, it can all be download at once. Qualitatively, Web IMDJ reduces the time required for preservation by approximately 30 min–1 h.
Comparison of the whole process of Table and Web IMDJs
Table IMDJ process
Web IMDJ process
Preparation for workshop phase
Determine the purpose and theme of the workshop
Decide DJs to use
Disassemble contents of DJ into sentence elements and output in CSV format
Import CSV file into a visualization tool
Visualize a map by using KeyGraph algorithm
Clean the visualized map for smooth discussion
Print the visualized map on large size sheets
Create DJ cards
Input necessary information of the workshop
Decide DJs to use
Visualize a map automatically
Clean the visualized map for smooth discussion
Arrange the date and place for the workshop
Set up the prepared implements
Inform about the ID and password
Convert all knowledge to digital data
Click the download button
Comparison Experiment of Table and Web IMDJs
The purpose of the experiment was to compare Table and Web IMDJs with respect to the number and quality of the proposed solutions as well as the knowledge-creation process.
The subjects included 32 Japanese industrial workers interested in data utilization. We allowed subjects to conduct both Table IMDJ and Web IMDJ workshops on different themes. We include working people because they have a strong inclination to solve social issues with their data. Note that a regular IMDJ workshop uses 25–30 DJs and takes approximately 90 min with 7 or 8 people; however, these conditions put a heavy burden on subjects. Furthermore, the communication among 7–8 people is too complicated to address as a knowledge-creation process. Therefore, in this experiment, we used 10 DJs and allow 5 min for raising requirements, 10 min for proposing solutions, and 3 min for evaluating solutions with two people per workshop. As a result, we could finish the whole experiment within 1 h.
Grouping to avoid affecting the workshop’s order and theme
Table IMDJ theme: city and transportation
Web IMDJ theme: tourism and consumption
Table IMDJ theme: tourism and consumption
Web IMDJ theme: city and transportation
Web IMDJ theme: city and transportation
Table IMDJ theme: tourism and consumption
Web IMDJ theme: tourism and consumption
Table IMDJ theme: city and transportation
The workshop themes were determined as “city and transportation” and “tourism and consumption” based on the words often included in search queries entered in the DJ Store , a platform for searching DJs. The DJ Store was considered because those who visit DJ Store are interested in data utilization, and search queries entered by them represent the fields of their interest. After determining the theme, we selected the 10 DJs to use in the workshop from the DJs searched for in the DJ Store using the themes as search queries.
On the day of the experiment, we delivered lectures to the subjects about IMDJ if they were not familiar with it. Thus, subjects understood IMDJ as a workshop method for accelerating data utilization before attending the workshop. When experimenting, we prepared a manual, which the practitioner used in proceeding with the experiment.
Evaluation index of each solution with detailed explanation
The extent of newness of a solution
Can we expect a sufficient market size?
Degree of possibility to realize the solution
Degree of usefulness of the solutions
Comparison of the Number and Quality of Proposed Solutions
As described earlier, 16 teams participated in the experiment; however, we did not use the data of 5 teams because of operational mistakes on our end and the fact that some subjects performed other work not relating to the experiment during the experiment (it is necessary to prevent external influences to the order and theme of the workshop). Thus, we used the data of 8 teams after excluding the 3 teams who performed the experiments last. In Web IMDJ, we determined some incorrect operations derived from the interface, for example, some ideas were almost the same or not associated with other ideas. In this study, we corrected such obvious mistakes in a way that the subjects originally intended. However, as a matter of course, it is essential to improve the interface so as not to allow such preventable mistakes to occur.
Moreover, as mentioned earlier, we acquired the evaluation of solutions from the subjects; however, this evaluation is insufficient as subjects are likely to be biased. Therefore, we asked students and researchers (hereafter, referred to as the third-party) who did not participate in the workshop and are familiar with the process of IMDJ to evaluate the solutions. We selected those familiar with IMDJ as evaluators because it seemed difficult to ask individuals to correctly evaluate the solutions if they did not know the invention process. The evaluation method was the same as the subject’s evaluation, i.e., a five-grade subjective evaluation with respect to the four indicators of novelty, marketability, feasibility, and utility. These four indicators were derived from past research which evaluates the scenarios based on solutions in novelty, feasibility, and utility . Here, we added marketability as an indicator.
The evaluators included 14 people, each of whom evaluated 19 or 20 solutions. The number of evaluations is 109 in the subject-based evaluation and 276 in the third-party-based evaluation. Note that the solutions by the subjects and third-party differed in that the former evaluate solutions by considering the discussion during the workshop while the latter evaluated solutions only according to the written pieces of contents.
Total amount of the proposed knowledge
Comparison of subject-based evaluation scores
Comparison of third-party-based evaluation scores
Comparison of the Knowledge-Creation Process
To eliminate ambiguity about elements, we explain this definition using an example. The title of DJ107 is railway congestion data that contain railway, congestion, and data as nouns. Here, “data” is the meaningless word because all DJs are about data. Then, let us suppose that the requirement “I want to avoid traffic congestion” is proposed in the workshop. As the DJ107 and the requirement have “congestion” in common, we add one to the shared element score between the DJ and requirements.
Total number of elements included in the discussion and the average of the number of elements included in the requirements (req), solutions (sol), and additional (add) DJs, per paper
Number of shared elements among requirements (req), solutions (sol), additional (add) DJs, remarks, and DJ in Table IMDJ
Number of shared elements among requirements (req), solutions (sol), additional (add) DJs, remarks, and DJ in Web IMDJ
In this section, we discuss the results of the experiment described in “Comparison of the number and quality of proposed solutions” and “Comparison of the knowledge-creation process” sections detail.
Tables 7 and 8 show the subject- and third-party-based evaluations, respectively. As described earlier, a difference exists because the subjects evaluate solutions by considering discussion during the workshop, whereas third-party members evaluate solutions only according to the written contents. In other words, the effect of discussions during the workshop will appear as differences in evaluations. Note that as shown in Table 9, the number of elements in remarks is much more significant in Table IMDJ. Thus, the influence of the discussion appears to be notable in the Table IMDJ. In Tables 7 and 8, marketability and utility scores of the Table IMDJ are much higher in subject-based evaluation than those in the third-party-based evaluation; however, these scores in Web IMDJ are only slightly different. Therefore, active discussion in Table IMDJ seems to be a critical factor of evaluation, especially with regard to marketability and utility.
At the same time, in Table 8, the scores of marketability, feasibility, and utility in Web IMDJ are equal to or higher than those in Table IMDJ. This result suggests that for written contents stored as data-utilization knowledge, Web IMDJ achieves a similar or higher score compared to that of Table IMDJ.
Concerning the knowledge-creation process, as argued in “Comparison of the knowledge-creation process” section, subjects actively held discussions in the Table IMDJ. Additionally, subjects hold discussions based on the contents and extracted keywords of DJs. These results are similar to the study by Abrams et al. , in which the authors argued that discussions by chat not only contain less information than face-to-face discussions but also often have nothing to do with the agenda.
In contrast, for IM, the predecessor of IMDJ, ideas proposed in the online workshop are superior in novelty and feasibility . Further, subjects invented more ideas in an online workshop than in a face-to-face workshop . Except for the novelty indicator, which we cannot evaluate effectively, these trends did not appear in our results.
These gaps result from the difference in the way subjects worked in IM and our research. In IM, subjects discussed to evaluate the proposed ideas. By contrast, the time for evaluation was too short in our experiment, such that subjects did not hold discussions during the evaluation. These differences in experimental procedures were considered to have a greater impact on the quality of solutions in Web IMDJ because there is a tendency for the creation and the evaluation of solutions to be separated due to restrictions in screen size. If we had taken more time for evaluation, not only the quality of solutions, but also the amount of conversation in Web IMDJ, could be larger.
From the viewpoint of divergence and convergence of discussion, the discussion in Table IMDJ is based on the DJ. Namely, subjects hold discussions without much divergence from the content of the DJ. However, according to Salter et al. , face-to-face discussion was so wide-ranging and loosely structured that divergent aspects of the topic were uncovered, whereas topics were explored more thoroughly in the online discussion. The reason for the resulting gap is that IMDJ focuses on inventing data-utilization knowledge, whereas reflective discussion allows for free discussion. In other words, IMDJ requires the convergence of the discussion as solutions, whereas reflective discussion focuses on practicing discussions.
To summarize, this experiment yields that the solutions in Table IMDJ were well considered based on active discussions, such that the data-utilization knowledge gained higher quality. However, the storage of all the discussions as data is cumbersome, and it is difficult to save them as reusable knowledge. On the contrary, in Web IMDJ, the proposed knowledge and discussions by chat can automatically be stored. Additionally, regarding the written contents on the sticky notes that are preserved as knowledge, Web IMDJ is equal to or better than Table IMDJ.
Regarding the knowledge-creation process, subjects held discussions based on the DJs, and the proposed solutions reflected the discussion in Table IMDJ. In contrast, in Web IMDJ, subjects invented solutions referring to other proposed ideas. Considering the comparison of these outcomes with regard to face-to-face and online communications, some earlier studies reported similar results; whereas, others reported different results. Contrastingly, we argue that the factor lies in the process or purpose of the communication.
Here, referring to the results and considerations, how should we proceed with data utilization in an IMDJ workshop? Although the number and quality of solutions in Web IMDJ are equal or superior to those in Table IMDJ with regard to the written contents, subjects actively conduct discussions in Table IMDJ. Thus, we should not conclude that it is better to only use Web IMDJ. Therefore, in the next section, we discuss in detail the most effective IMDJ operation method for promoting data utilization by analyzing the acquired data.
Most effective IMDJ operation method for promoting data utilization
In the previous section, we compared Table and Web IMDJs in terms of the number and quality of the solutions, and the knowledge-creation process. In this section, by analyzing the data acquired in the workshop in detail, we propose the most effective IMDJ operation method for promoting data utilization. Specifically, we consider the relationship between the number of combined DJs and the evaluation score, the influence of workshop order, and subjects’ opinions.
Relationship between the number of combined DJs and evaluation score
Spearman’s rank correlation coefficient for the number of DJs related to the solution and evaluation scores
The third party
Influence of the workshop order
Number of included elements in each team (classified by groups of Table 4)
Elements in discussion
Comparison of evaluation scores of the teams holding heavy and least amount of discussions in Table IMDJ
Rich discussion (Table → Web)
Small discussion (Web → Table)
Comparison of evaluation scores of the teams holding heavy and least amount of discussions in Web IMDJ
Rich discussion (Table → Web)
Small discussion (Web → Table)
Comparison of the number of proposed solutions of the teams holding heavy and least amount of discussions
Rich discussion (Table → Web)
Lean discussion (Table → Web)
Subject opinion questionnaire
Please describe what you noticed and felt about the differences between Table IMDJ and Web IMDJ
Comments for Table IMDJ
Easy to overlook and to lead letters thoroughly
Easy to get excited by sharing non-verbal information
Unlikely to miss the points thanks to oral communication
Comments for Web IMDJ
Easy to put out many ideas
Visualized map and sticky notes are easy to see, but it takes time to get used to
Feel free to link a solution with DJs
I can concentrate on my thoughts, but the sense of making solutions by discussion is weak. Feel like doing separate work
I concentrated on my thoughts
Please describe what you feel is convenient or inconvenient for Web IMDJ
Able to link requirement, solution, DJ with drag and drop operation
Accommodate the flexibility from the full bird’s eye view to detailed browsing by scaling
Fundamentally useful as a tool
Multiple people can collaborate simultaneously at a distance
Remain as data
Linking operation is hard, and the map is confusing to see (color scheme, overlapping of characters and lines, etc.)
Hard to start without explanation. Need materials for immediate operation
Hard to operate when the screen is small
Once expanded, the characters of the map remained large even if I shrink it
Hard to grasp the information when many solutions appear
Hard to notice that others stuck the ideas
Even if the requirements and solutions are linked, it does not visualize on the map
Need to change the font size manually
I could not talk until I went to a specific chat room and I was at a loss where to talk the idea noticed on the spot. I felt they affected the progress
Please describe the desired or required functions for Web IMDJ in promoting data utilization
3D network diagram
Visualize the connection between DJs and solutions
Auxiliary function to reduce the burden of creating scenarios such as scenario planning framework
In addition to chat, need voice chat as used in a web meeting
Function to automatically convert the input voice into text and display it
Stepless zoom function
Change chat notification from displaying red circle to changing the shape of tabs
Notification when a sticky note appears
Function to check unidentified sticky note
Always show the chat. Mention function
No chat function required
Function for quickly reaching the sticky notes I like, such as “Favorites” or “Watch”
We have two purposes for acquiring this questionnaire. One is to argue the difference between Table and Web IMDJs from the viewpoint of the subject, which corresponds to the first question. For example, we received answers “easy to get excited by sharing nonverbal information” in Table IMDJ and “I concentrate on my thoughts” in Web IMDJ. These opinions seem to represent contributing factors for the rich discussion in Table IMDJ versus the small discussion in Web IMDJ. However, as these opinions were much affected by individual differences, we only considered qualitative evaluation.
Another purpose is to examine the method of improving the interface of Web IMDJ from the opinion of subjects who experienced Web IMDJ; this purpose, answers the second and third questions. Among various answers, there is a high demand for improvements on the interface related to the association between DJs and solutions, and communication.
Table 12 shows a correlation between the number of combined DJs in the solution and the feasibility and utility. This is because a combination of multiple DJs makes it easier to propose cross-domain solutions, resulting in a high score in utility. In other words, the small number of combined DJs in Web IMDJ tends to lower the solution quality because sufficient information cannot be reflected in the solution. Referring to the subject’s opinion, the small number of combined DJs can be attributed to the problem in the interface of associating DJs and solutions. Thus, by improving the interface, the quality of the proposed solution increases in Web IMDJ.
Considering the workshop order, the performance of the second workshop would be superior because of the experience gained in the first workshop, as shown in Table 15. In other words, for the solution proposed in Web IMDJ, the evaluation scores of marketability, feasibility, and utility are superior to the team that first used Table IMDJ. By contrast, Table 14 shows that the marketability score in Table IMDJ is almost the same, and the score of feasibility is higher for the team who first used Table IMDJ. This result could be because of the vibrant discussion in Table IMDJ.
However, regarding the novelty, the teams that attended the Web IMDJ workshop first gave high evaluations for both Table and Web IMDJs. This result suggests that small discussion leads to the invention of new ideas. Additionally, Table 16 shows that the number of proposed solutions is higher for teams with lean discussions. Thus, although teams that first conducted Web IMDJ hold fewer discussions, they propose more solutions.
To summarize, the analysis results show that when we start from Table IMDJ, we can actively discuss and invent solutions superior in marketability and feasibility, as discussed in “Discussion” section. Here, the indicators in which differences appear are different because the number of targeted teams is as small as three. However, what is essential here is that the activeness of discussion affects the quality of the proposed solution, and thus it is necessary to improve the interface of Web IMDJ by referring to the studies to activate online discussions .
In contrast, when users start from Web IMDJ, they have fewer discussions, but propose more solutions, which are superior in terms of the score of novelty. This suggests that when we hold fewer discussions, we have less resistance to proposing idea-based solutions and can invent new solutions.
How is the influence of order of face-to-face and online communications argued in other fields? According to some subjects in the field of data research, participants are asked to communicate online before they perform group work face-to-face, for smooth progress of experiments. Additionally, in the field of medical care, there is a recognition that online communication makes it easier to distinguish the opinions of participants . These results conflict with the result that subjects discuss actively when they first use Table IMDJ. This conflict is attributed to the difference in the purpose of the disciplines; that is, in the field of data research and medical care, communication itself is essential; whereas, inventing data-utilization scenarios is essential in IMDJ.
Based on the abovementioned discussions, we can propose an effective IMDJ operation method according to the given purpose: when we want to hold more discussions and acquire superior solutions in marketability, feasibility, and utility, we use Table IMDJ first; in contrast, when we want to acquire brand-new solutions in a casual atmosphere, we use Web IMDJ first. However, in general, IMDJ workshop essentially requires discussion among stakeholders, and thus, the activation of discussion is desired for overcoming the contextual gap among the different domains of stakeholders.
Therefore, it is better to start with Table IMDJ. It is also true that conducting a Table IMDJ workshop is cumbersome. Moreover, as argued in “Discussion” section, Web IMDJ allows inventing solutions equal or superior to those of Table IMDJ with regard to the written content stored as data-utilization knowledge. Consequently, by continuously holding discussions in Web IMDJ after the first Table IMDJ workshop, we can effectively progress data utilization.
An important opinion of the subjects is that Web IMDJ still has room for improvement. For example, a subject suggested adding voice chat, which is easy to achieve by integrating other apps. Here, recall that Abrams et al.  reported that the data richness of online audiovisual focus groups is the same as that of face-to-face focus groups. This indicates that enabling voice chat activates discussions in Web IMDJ. Therefore, Web IMDJ has the potential to further contribute to the promotion of data utilization by improving the interface.
Overall, here we have outlined the most efficient IMDJ operation method. That is, after improving the critical interface issues as soon as possible, a Table IMDJ workshop must first be conducted, followed by continuing discussions on Web IMDJ.
To promote data utilization, we implemented Web IMDJ, a platform for discussing the invention of data-utilization scenarios. To examine the influence of different communication media, namely face-to-face or online methods, we compared both Table and Web IMDJs in terms of various aspects. Experimental results showed that the number and quality of solutions in Web IMDJ are equal or superior to those in Table IMDJ, in terms of the written contents; however, subjects actively discussed more in Table IMDJ. Further, we proposed the most efficient IMDJ operation method by combining the features of Table and Web IMDJs.
The contribution of this paper can be divided into two main aspects. First, we promote data utilization. Our platform, Web IMDJ, enables us to repeat the workshop in which we invent the data-utilization scenarios to be as useful as those in Table IMDJ, with lesser complications. Second, we compared the difference in the communication media with discussions on data utilization. As IMDJ is a complicated workshop method for inventing new knowledge, our study can provide suggestions into this research field.
In the actual discussions for data utilization, there are cases in which Web or Table IMDJ can not only be used independently, but also together, for better discussions. In such cases, it is necessary to expand the settings of our study to assess the quality of facilitation and communication.
We have been developing a platform integrated with the DJ site, DJ Store, and Web IMDJ, all of which are currently separate tools. Once completed, the platform will improve user experience and accelerate the accumulation of data-utilization knowledge. In the future, utilizing the accumulated data-utilization knowledge can inform determinations of proper data prices.
https://every-sense.com/eng/. Accessed June 2019.
http://j-dex.co.jp/en/index.html. Accessed June 2019.
https://azuremarketplace.microsoft.com/en-us. Accessed June 2019.
https://ckan.org/. Accessed June 2019.
https://www.qlik.com/us/. Accessed June 2019.
https://www.dawex.com/en/. Accessed June 2019.
https://www.sakura.ad.jp/information/pressreleases/2017/12/05/90200/. Japanese only. Accessed June 2019.
Press release, Japanese only. Accessed June 2019. https://pr.fujitsu.com/jp/news/2018/05/14-2.html.
Press release, Japanese only. Accessed June 2019.
https://datajacket.org/?lang=english. Accessed June 2019.
https://imdj.datajacket.org/. Accessed June 2019.
This work has been supported by industrial collaborators of Ohsawa Laboratory, including Kozo Keikaku Engineering Inc. The authors would like to thank all the staff members of Kozo Keikaku Engineering Inc., and FUJITSU LIMITED for supporting the research; and Editage (www.editage.com) for English language editing.
This study was partially supported by JST CREST [No. JPMJCR1304], JSPS KAKENHI [JP16H01836 and JP16K12428].
Compliance with Ethical Standards
Conflict of interest
The authors declare that they have no conflict of interest.
- 2.Xing, B.: Creativity and artificial intelligence: a digital art perspective (2018). https://doi.org/10.2139/ssrn.3225323
- 4.Rittinghouse, J.W., Ransome, J.F.: Cloud Computing: Implementation, Management, and Security. CRC Press, Boca Raton (2016)Google Scholar
- 6.Misbahuddin, S., Zubairi, J.A., Saggaf, A., Basuni, J., A-Wadany, S., Al-Sofi, A.: IoT based dynamic road traffic management for smart cities. In: 2015 12th International Conference on High-capacity Optical Networks and Enabling/Emerging Technologies (HONET), Islamabad, pp. 1–5 (2015)Google Scholar
- 13.Stahl, F., Schomm, F., Vomfell, L., Vossen, G.: Marketplaces for digital data: Quo vadis? (No. 24). Working Papers, ERCIS-European Research Center for Information Systems (2015)Google Scholar
- 15.Zhang, M., Arafa, A., Huang, J., Poor, H. V.: How to price fresh data. (2019). arXiv preprint arXiv:1904.06899
- 16.Mano, H.: EverySense: An end-to-end IoT market platform. In: Adjunct Proceedings of the 13th International Conference on Mobile and Ubiquitous Systems: Computing Networking and Services (MOBIQUITOUS 2016). ACM, New York, NY, USA, pp. 1–5 (2016)Google Scholar
- 17.Chappell, D.: Introducing the windows azure platform. David Chappell & Associates White Paper (2010)Google Scholar
- 18.Winn, J.: Open data and the academy: an evaluation of CKAN for research data management. In: IASSIST 2013, Cologne (2013)Google Scholar
- 19.Ponte, D.: Enabling an Open Data Ecosystem. ECIS 2015 Research-in-Progress Papers, Paper 55 (2015)Google Scholar
- 25.Zheng, Z., Peng, Y., Wu, F., Tang, S., Chen, G.: An online pricing mechanism for mobile crowdsensing data markets. In: Proceedings of the 18th ACM International Symposium on Mobile Ad Hoc Networking and Computing, p. 26 (2017)Google Scholar
- 26.Niyato, D., Alsheikh, M. A., Wang, P., Kim, D. I., Han, Z.: Market model and optimal pricing scheme of big data and Internet of Things (IoT). In: 2016 IEEE International Conference on Communication (ICC), Kuala Lumpur, pp. 1–6 (2016)Google Scholar
- 27.Sen, S., Joe-Wong, C., Ha, S., Chiang, M.: Smart data pricing (SDP): Economic solutions to network congestion. Rec. Adv. Netw. 1, 221–274 (2013)Google Scholar
- 28.Cao, X., Chen, Y., Liu, K.J.R.: An iterative auction mechanism for data trading. In: IEEE International Conference on Acoustics, Speech, Signal Processing (ICASSP), New Orleans, LA, pp. 5850–5854 (2017)Google Scholar
- 29.Susanto, H., Zhang, H., Ho, S., Liu, B.: Effective mobile data trading in secondary ad hoc market with heterogeneous and dynamic environment. In: IEEE 37th International Conference on Distributed Computing Systems (ICDCS), Atlanta, GA, pp. 645–655. IEEE (2017)Google Scholar
- 31.Zheng, Z., Peng, Y., Wu, F., Tang, S., Chen, G.: ARETE: on designing joint online pricing and reward sharing mechanisms for mobile data markets. IEEE Trans. Mobile Comput. (2019). https://doi.org/10.1109/TMC.2019.2900243
- 33.Ohsawa, Y., Kido, H., Hayashi, T., Liu, C., Komoda, K.: Innovators marketplace on data jackets, for valuating, sharing, and synthesizing data. In: Knowledge-Based Information Systems in Practice, pp. 83–97. Springer, Cham (2015)Google Scholar
- 35.Ejiri Y., Ikeda E, Sasaki H.: Realization of data exchange and utilization society by Blockchain and Data Jacket: merit of consortium to accelerate co-creation. In: IEEE International Conference on Data Mining Workshops (ICDMW), pp. 180–182. IEEE (2018)Google Scholar
- 36.Hayashi, T., Ohsawa, Y.: Retrieval system for data utilization knowledge integrating stakeholders’ interests. In: AAAI Spring Symposium 2018 Beyond Machine Intelligence: Understanding Cognitive Bias and Humanity for Well-being AI (2018)Google Scholar
- 37.Ohsawa, Y., Benson, N. E., Yachida, M.: KeyGraph: automatic indexing by co-occurrence graph based on building construction metaphor. In: Proceedings IEEE International Forum on Research and Technology Advances in Digital Libraries (ADL’98), pp. 12–18. IEEE (1998)Google Scholar
- 38.Hayashi, T., Ohsawa, Y.: Knowledge structuring and reuse system design using RDF for creating a market of data. In: 2nd International Conference on Signal Processing and Integrated Networks (SPIN), pp. 607–612. IEEE (2015)Google Scholar
- 39.Beckett, D., McBride, B.: RDF/XML syntax specification (revised). W3C Recommendation 10.2.3 (2004). http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.454.3972&rep=rep1&type=pdf
- 40.Allen, I.E., Seaman, J.: Online Report Card: Tracking Online Education on the United States. Babson Survey Research Group; Quahog Research Group, LLC Alfred P. Sloan Foundation (ERIC Document Reproduction Service No. ED572777) (2016)Google Scholar
- 42.Powell, A., Watson, J., Staley, P., Patrick, S., Horn, M., Fetzer, L., Hibbard, L., Oglesby, J., Verma, S.: Blending Learning: The Evolution of Online and Face-to-Face Education from 2008–2015. Promising Practices in Blended and Online Learning Series. Michigan; Nevada; New York; North Carolina; Pennsylvania; Utah; Washington (ERIC Document Reproduction Service No. ED560788 (2015)Google Scholar
- 43.Olthuis, J.V., Watt, M.C., Bailey, K., Hayden, J.A., Stewart, S.H.: Therapist-supported Internet cognitive behavioural therapy for anxiety disorders in adults. Cochrane Database Syst. Rev. 3, CD011565 (2016)Google Scholar
- 48.Ohsawa, Y., Okamoto, K., Takahashi, Y., Nishihara, Y.: Innovators marketplace as game on the table versus board on the web. In: 2010 IEEE International Conference on Data Mining Workshops, pp. 816–821. IEEE (2010)Google Scholar
- 51.Fette, I., Melnikov, A.: The WebSocket Protocol. RFC 6455, December 2011. https://www.rfc-editor.org/info/rfc6455
- 53.Chodorow, K., Mongo, D.B.: The Definitive Guide: Powerful and Scalable Data Storage. O’Reilly Media, Newton (2013)Google Scholar
- 57.Ito, T., Shibata, D., Suzuki, S., Yamaguchi, N, Nishida, T., Hiraishi, K., Yoshino K.: Agent that facilitates crowd discussion. In: 7th ACM Collective Intelligence 2019, Carnegie Mellon University, Pittsburgh, USA, June 13–14 (2019)Google Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.