1 Introduction

Modes of human communication are classified into two general categories: face-to-face communication and remote communication [1]. In recent years, the frequency of remote communication via email, video calls, and other media has greatly increased due to the proliferation of the internet and smartphones. In face-to-face communication, just as the speaker’s verbal information is transmitted, so is non-verbal information such as both the speaker’s and listener’s facial expression, body language, and tone of voice. At the same time, methods of remote communication, such as text chat, are limited in the amount of non-verbal information they provide, and consequently pictographs and emoticons are used to compensate for this deficiency.

The importance of non-verbal information in communicating has been explored in numerous studies, and non-verbal information is believed to be essential to understanding conversational content [2, 3],. Conversely, research that is skeptical of this importance has also been conducted. Kimura and Tsuzuki suggested that text-only CMC (Computer Mediated Communication) is easier to communicate with than face-to-face methods as a result of CMC acting as a sort of filtration system that eliminates non-verbal information such as eye contact and gestures present when face-to-face [4]. In another study, Sugitani examined the communicability of information from two perspectives: the degree of transmission and the feeling of transmission [5]. The results of Sugitani’s experiment suggest that the communicability of information should be analyzed using these two perspectives separately.

2 Non-verbal Information and Feeling and Degree of Transmission

2.1 Importance of Non-verbal Information

65% to 93% of a message is delivered by non-verbal information when communicating, as hypothesized by multiple researchers [1]. Sugitani’s research indicated that among various forms of communication, most people felt communicating information was easiest face-to-face [5]. Furthermore, approximately 90% of these people gave the fact that they can see their partner’s facial expression as the reason. These results indicate that many people believe in the importance of non-verbal information in communication.

2.2 Non-verbal Information in Communication Modes

Table 1 shows the non-verbal information exchanged in each communication mode. The amount and type of non-verbal information exchanged varies based on the mode. However, the specific advantages of each communication mode—whether it be face-to-face, email, chat, video calls, etc.—have not been confirmed [6].

Table 1. Non-verbal information exchanged in each mode of communication
Full size table

2.3 Related Studies

Sugitani examined the communicability of information with two perspectives: the degree of transmission and the feeling of transmission [5]. The degree of transmission indicates the extent that a speaker’s dialogue is correctly received by their listener, and the feeling of transmission indicates the extent that the speaker and listener feel that the information was correctly shared. The experiment results showed that text chat and voice chat had lower feelings of transmission than face-to-face communication, but also had higher degrees of transmission. In other words, this suggests that the feeling of transmission increases with an increase in non-verbal information, as shown in Fig. 1, but the degree of transmission may actually decrease. However, the only differing factor between face-to-face communication, voice chat, and text chat is the characteristics of their non-verbal information. Therefore, it is crucial to test this trend by altering the amount of the particular non-verbal information in each mode of communication.

Fig. 1.
figure 1

Relationship between amount of non-verbal information and feeling/degree of transmission

Full size image

2.4 Study Objectives

This study aimed to more thoroughly test the effects of non-verbal information on feeling and degree of transmission by controlling and altering communication mode and the amount of visual non-verbal information relayed.

3 Effects of Communication Mode on Feeling and Degree of Transmission

3.1 Outline

Based on Sugitani’s study, this study tested the effects of non-verbal information on feeling and degree of transmission among three communication modes: text chat, voice chat, and face-to-face.

3.2 Verification Experiment

Experimental Method

Figure 2 shows the conditions of the experiments. The participants were paired into groups of two with one person taking the role of speaker and the other listener. The speaker then relayed to the listener a story they were told. This task was performed once via text chat, once via voice chat, and once face-to-face.

Fig. 2.
figure 2

Experimental conditions I

Full size image

Figure 3 shows the experimental procedure. In the experiment, i. the participant with the role of speaker memorized a story, ii. relayed that story to listener, and iii. a confirmation test and questionnaire were given for each of the three segments (text chat, voice chat, and face-to-face communication). In addition, as preliminary questionnaires, the two participants took KiSS-18 evaluations and were made to answer questions regarding the pair’s connection and compatibility. KiSS-18 is a scale that measures a person’s mastery of social skills, and its high degree of reliability and validity has been demonstrated [7, 8].

Fig. 3.
figure 3

Experimental procedure

Full size image

Table 2 shows the questionnaire fields. The questionnaire contained fields regarding the climate of the dialogue and impressions of the story’s characters that participants rated on a scale of one to seven.

Table 2. Questionnaire fields
Full size table

The confirmation test investigated the listener on the story’s keywords given to the speaker at the beginning of the experiment. The test format was multiple choice instead of a free writing format so as to prevent the researcher’s subjectivity from affecting the grading of the test.

Evaluation Method

Degree of transmission was evaluated through the accuracy of the transmitted details and sentimental information. The accuracy rates of the listeners’ confirmation test results were used to evaluate the degree of transmission of the story’s contents. This accuracy rate is the ratio of the number of questions that the listener answered correctly out of the number of questions whose keywords were judged by the researcher to be transmitted correctly by the speaker. In addition, the questionnaire results regarding the impressions of the story characters were used to evaluate the degree of transmission of the sentimental information, and the degree of consistency in impressions was measured. Specifically, the absolute value was taken of the difference between the speaker’s and listener’s impressions of three story characters, and these values were compared among the three experimental conditions. A higher number indicates a higher discrepancy between the speaker’s and listener’s impressions. Sense of transmission was evaluated using the questionnaire results regarding the dialogue content.

Experimental Results

Figure 4 shows the accuracy rate results from the listeners’ confirmation tests. Little difference was observed between the text chat and voice chat results, and while the face-to-face communication results were slightly lower, no statistically significant difference was observed. Figure 5 shows the absolute value of the difference between the speakers’ and listeners’ impressions of the story characters. Face-to-face communication demonstrated a greater gap between the speakers’ and listeners’ impressions than text chat and voice chat.

Fig. 4.
figure 4

Accuracy rate of listeners’ confirmation tests

Full size image
Fig. 5.
figure 5

Absolute value of difference between speakers’ and listeners’ impressions of characters

Full size image

Figure 6 shows the combined results of the speakers’ and listeners’ questionnaires. Feeling of transmission is believed to be heavily influenced by the ease of communicating or listening and the participant’s feeling of connection with their partner. Question 1 showed no statistically significant difference, but other questions did. Therefore, it is believed that feeling of transmission was rated highest for face-to-face communication, followed by voice chat, and lastly by text chat.

Fig. 6.
figure 6

Questionnaire results

Full size image

Discussions

Table 3 shows the ranked results of the feeling and degree of transmission of the three communication modes. As the feeling of transmission and degree of transmission show different results, it is necessary to consider the communicability of information by treating these as two separate factors. Moreover, the characteristics of the non-verbal information present in the three modes of communication are fundamentally different. In order to verify the trends shown in Fig. 1, it is necessary conduct verification experiments that focus on one single mode of communication at a time and quantitatively control that communication mode’s non-verbal information.

Table 3. Evaluation of feeling/degree of transmission for each communication mode
Full size table

4 Effects of Visual Non-verbal Information on Feeling and Degree of Transmission

4.1 Outline

Based on the analysis in the previous section, this section focuses on the face-to-face mode of communication and outlines the proposed method for the quantitative controlling of visual non-verbal information. The effects of visual non-verbal information on the feeling and degree of transmission in face-to-face communication were tested using this proposed method.

4.2 Control Method for Visual Non-verbal Information

Proposed Method

In the face-to-face communication mode, the sender and receiver of non-verbal information sat across from each other and were separated by fabric. It was hypothesized that the amount of visual non-verbal information passed on to the receiver can be controlled quantitatively by increasing or decreasing the number of sheets of fabric, and the validity of this hypothesis was tested in an evaluation experiment.

Evaluation Experiment

In the experiment, participants were shown through the fabric a total of 12 videos of person displaying facial expressions, body gestures, and hand gestures. Participants were made to answer the question, “Which facial expression (or head/hand gesture) is the person in the video making?” for each video. The videos contained no sound.

Figure 7 shows the experimental conditions. White sheets of tulle fabric measuring 200 cm height by 115 cm width with denier 50 were hung halfway between the participants and the video. By increasing or decreasing the number of sheets of fabric, the amount of visual non-verbal information transmitted was altered. A preliminary experiment was performed to determine the appropriate number of sheets of fabric, leading to the decision of seven patterns. The order of the videos and experimental conditions was randomly changed to create 14 patterns in which the experiment was performed. The experiment was evaluated and the seven experimental conditions were compared using the accuracy rate of answers to the test questions. The test format was multiple choice instead of a free writing format so as to prevent the researcher’s subjectivity from affecting the grading of the test.

Fig. 7.
figure 7

Experimental conditions II

Full size image

Experimental Results

Figure 8 shows the average accuracy rate of answers to the confirmation test questions. The Bonferroni method was employed to assess the results. The accuracy rate for four sheets of fabric and eight sheets of fabric was almost 100%, and statistically significant differences of 95% and 99% were observed for all other numbers of sheets of fabric. Additionally, the accuracy rate decreased by approximately 20% as the number of sheets of fabric increased by two for each experimental condition beyond eight sheets. Therefore, by using the proposed method, it was evident that the amount of visual non-verbal information relayed can be controlled incrementally. In addition, average accuracy rates were tested by separating facial expressions and gestures (hand and head movements). Results indicated that 12 sheets of fabric was a unique experimental condition in which facial expressions were virtually impossible to read, but gestures could be sufficiently read.

Fig. 8.
figure 8

Accuracy rate of confirmation test on video content

Full size image

4.3 Verification Experiment

Experimental Method

Using the method explained previously, a verification experiment was performed to investigate the effects of visual non-verbal information on feeling and degree of transmission. Participants were paired into groups of two. The tasks performed and the evaluation criteria were the same as those outlined in the previous section. The number of sheets of fabric were increased or decreased to 8 sheets, 10 sheets, 12 sheets, 14 sheets, and 16 sheets to conduct the telling of the story, done one at a time. The experimental conditions (number of sheets) was determined using the results outlined in the previous section. However, in order to reduce the burden on participants, each was made to perform one pattern containing only three different experimental conditions: either Pattern A (8 sheets, 10 sheets, and 14 sheets) or Pattern B (8 sheets, 12 sheets, and 16 sheets).

Experimental Results and Analysis

While there were two patterns, Pattern A and Pattern B, both patterns shared the eight-sheet segment, and the results of the eight-sheet segment were used as the value of 1 in the figures and proportions explained below and were also used to integrate the results of both patterns.

Figure 9 shows the proportion of the listeners’ accuracy rates, and Fig. 10 shows the proportion of the absolute values of the difference in impressions of the story characters. Neither figure shows statistically significant differences, but although the listeners’ accuracy was highest for the 12-sheet segment, the same segment shows the greatest difference in impressions of the story characters. Therefore, there is a possibility that the degree of transmission of information regarding the dialogue content and the degree of transmission of sentimental information show opposite trends. These results suggest that the total amount of verbal and non-verbal information that can be received and interpreted may be limited for each individual.

Fig. 9.
figure 9

Proportion of listeners’ accuracy rate

Full size image
Fig. 10.
figure 10

Proportion of absolute values of difference in impressions of story characters

Full size image

Figure 11 shows the proportions of the speakers’ and listeners’ answers to the questionnaire. The speakers’ feeling of transmission in the 12-sheet segment (Question 1) was rated low, and their ease and enjoyment when communicating (Questions 2 and 4) in the 10-sheet segment was rated high. Conversely, the listeners’ ratings for the questionnaire fields of the 12-sheet segment were all high. It cannot be concluded for certain as the participants’ experiment patterns (Pattern A and Pattern B) differed, but it may be easiest for the speaker to communicate when they can somewhat see their partner’s facial expression, and it may be easiest for the listener to listen when they can see their partner’s gestures but not their facial expressions.

Fig. 11.
figure 11

Proportions of speakers’ and listeners’ questionnaire answers(Left: speakers, Right: listeners)

Full size image

Table 4 shows the rankings of the five experimental conditions for feeling and degree of transmission which are FoT and DoT. The speakers’ feeling of transmission displayed a strong positive correlation with consistency in character impressions. On the other hand, the listeners’ feeling of transmission displayed a strong positive correlation with the listeners’ accuracy rates. However, feeling and degree of transmission did not show a proportional correlation with the non-verbal information shown in Fig. 1.

Table 4. Rankings of each experimental condition for feeling/degree of transmission
Full size table

5 Conclusion

This study investigated the effects of non-verbal information on the feeling and degree of transmission by controlling the visual non-verbal information and communication mode. The experimental results of the three communication modes—text chat, voice chat, and face-to-face communication—showed that the degree of transmission was lowest in face-to-face communication as evaluated with the listeners’ test accuracy rates and consistency of character impressions. Conversely, according to the questionnaire results, feeling of transmission was ranked highest for face-to-face communication, followed by voice chat, and lastly text chat. These results suggested that the communicability of information should be considered using feeling of transmission and degree of transmission as two separate factors. Moreover, a negative correlation between the degree of transmission of the story content and consistency of story character impressions was observed in the experiment results performed by controlling visual non-verbal information. These results suggested that the total amount of verbal and non-verbal information that can be received and interpreted may be limited for each individual. It is conceivable that future research may need to be performed using tasks involving conversing freely or communicating to build agreement, or perhaps by controlling non-verbal information of different characteristics to further clarify the effects of non-verbal information on feeling and degree of transmission. Based on the results of such studies, it could be possible to arrange the most suitable forms of communication according to one’s aims by including or controlling specific non-verbal information in the future.