Introduction

Obstructive sleep apnea (OSA) is a sleep-related breathing disorder characterized by apneic (absent airflow) or hypopneic (reduced airflow) events that result from repetitive narrowing and/or collapsing of the upper airway and that commonly result in oxygen desaturations and arousals from sleep [1]. Individuals with OSA commonly have symptoms of morning headache, daytime sleepiness, and loud snoring [1]. The prevalence of OSA ranges from 2 to 14% in community-screened populations to a higher prevalence in certain subgroups (e.g., in men and in obese individuals) [2, 3]. OSA has been reported to be associated with many other health conditions such as stroke, hypertension, depression, diabetes, coronary artery disease, and sleep-related movement disorders, such as periodic leg movement during sleep and sleep bruxism (SB) [4,5,6].

SB is a masticatory muscle activity during sleep that is characterized by rhythmic masticatory muscle activity (RMMA) and may induce severe tooth wear, orofacial pain, and temporomandibular disorder [7]. Previous study has reported that OSA could be an independent risk factor for SB [8]. Furthermore, some studies reported that around half of individuals with OSA also have SB, suggesting that SB is a common comorbidity of OSA and that there is a close association between the two conditions [9, 10].

Although the underlying mechanism of the association between OSA and SB is still unclear, previous studies suggested that the occurrence of RMMA might be related to recurrent respiratory events and sleep arousals in OSA. Based on this, we hypothesized that effective OSA therapies will decrease the frequency of RMMA in individuals with OSA. This has been reported in two cases with concomitant OSA and SB [11, 12]. RMMA episodes in these two cases disappeared during continuous positive airway pressure (CPAP) therapy, while RMMA recurred when CPAP was removed [11, 12]. Similarly, other studies have demonstrated that mandibular advancement appliances (MAA) significantly reduced RMMA episodes in individuals with SB [13, 14]. However, few studies have been performed to investigate the effects of OSA therapies on SB in adults with OSA. Therefore, the first aim of this pilot study was to determine the effects of OSA therapies (CPAP and MAA) on SB in adults with OSA and in a subgroup of those with comorbid SB. We hypothesized that both CPAP and MAA therapies will significantly reduce RMMA in adults with OSA as well as in those with comorbid SB. In addition, since a growing number of studies indicated that the occurrence of RMMA was related to arousals rather than to respiratory events [15, 16], we hypothesized that both OSA therapies would significantly reduce RMMA related to respiratory arousals (RMMA-RAr), while having no effect on RMMA related to non-respiratory arousals (RMMA-nRAr). Therefore, the second aim of this pilot study was to investigate the effect of CPAP and MAA on the RMMA-RAr and RMMA-nRAr in individuals with OSA and in those with comorbid SB.

Methods

This cohort study is part of a large-scale prospective polysomnographic (PSG) study on the associations between SB and other sleep-related disorders. The protocol was approved by the institutional Medical Ethics Committee of the OLVG West, Amsterdam (WO 16–577). This study has also been registered on trialsearch.who.int (Trial NL8516).

Participants

The sample was collected on the basis of the medical history of patients who were referred to the Department of Clinical Neurophysiology, OLVG West, Amsterdam, the Netherlands, between April 2017 to July 2018. Individuals who met the following criteria were included in this study: 1) age ≥ 18 years; 2) diagnosed with OSA (baseline apnea–hypopnea index (AHI) ≥ 5 events/hour); 3) received treatment with CPAP or MAA; 4) received two PSG recordings, one without treatment and one with treatment; 5) the PSG recordings had bilateral masseter muscle electromyographic (EMG) traces [17]. Individuals were excluded: 1) with < 4 h of sleep during the recording; 2) with missing data on electroencephalography or masseter EMG; 3) with other sleep-related breathing disorders (e.g., asthma, chronic obstructive pulmonary disease), neurological disorders (e.g., epilepsy), or sleep-related movement disorders except for SB (e.g., periodic limb movement disorder); 4) received other OSA treatments, such as bariatric surgery, oral and maxillofacial surgery, and positional therapy.

Polysomnography and scoring criteria

Full night PSG recordings were performed by a portable compact PSG system (SOMNOscreen Plus, SOMNOmedics GmbH, Germany). The PSG system consisted of electroencephalography (F4C4, C4O2, F3C3, C3O1), electrooculogram (E1M2, E2M1), electrocardiogram, EMG (bilateral masseter muscles, anterior tibialis, mentalis and submentalis), pressure airflow, abdominal/thoracic respiratory effort, oxygen saturation, plethysmograph, heart rhythm, and sleep position. The mounting was performed by certified sleep technicians at the Department of Clinical Neurophysiology, OLVG West, Amsterdam, the Netherlands.

PSG recordings were analyzed offline using Domino software (SOMNOmedics GmbH, Germany). Sleep stages and respiratory events (viz., apnea, hypopnea, and respiratory effort-related arousal) were scored manually by sleep technicians from OLVG according to the American Academy of Sleep Medicine (AASM) scoring manual of sleep and associated events [18]. Sleep arousals were scored by two of us (D.L. and B.K.), following the scoring rules of AASM [18]. Sleep arousals were classified as respiratory arousals (RAr) or non-respiratory arousals (nRAr) [19]. Arousals occurring at the termination of respiratory events were defined as RAr, while those without preceding respiratory events were defined as nRAr.

The masseter EMG signals were filtered according to the AASM scoring manual (50 Hz notch; 10 Hz high pass; 100 Hz low pass) [18]. RMMA was scored according to previously reported criteria [17]. Each RMMA burst must exceed twice the amplitude of background EMG and be present simultaneously on the bilateral masseter muscles EMG traces. RMMA bursts occurring at an interval shorter than 3 s were regarded as a single episode. RMMA episodes were classified into three subtypes: phasic RMMA (three or more continuous RMMA bursts that are 0.25-2 s in the duration), tonic RMMA (one or more RMMA bursts ≥ 2 s), and mixed RMMA (if both phasic and tonic RMMA bursts are present within an episode). In addition, RMMA was considered to be related to arousal (RMMA-RAr or RMMA-nRAr) when they occurred within 5 s of an arousal [20].

Statistical analysis

The number of RMMA episodes was transformed into an index, defined as the number of events per hour of sleep. Individuals with an RMMA index of at least two episodes per hour of sleep were diagnosed with SB [17]. Individuals with concomitant OSA and SB were included in a subgroup for subgroup analysis. As the treatment durations of OSA therapies were not recorded in all participants’ medical history, this parameter was represented by the interval of the two PSG recordings for each patient.

The normality of outcome variables was assessed by using the Shapiro–Wilk test. Normally distributed data are presented as mean (standard deviation, SD). Non-normally distributed data are presented as quartiles (25%|median|75%). The comparisons of baseline characteristics of the two therapy groups, including sleep variables, respiratory variables, and treatment durations, were analyzed using the independent t-test (for normally distributed variables), Mann–Whitney U test (for non-normally distributed variables), or Chi-square test (for nominal variables).

Two-way repeated measures analyses of variance (ANOVA) were applied to analyze the mean difference of RMMA variables (i.e., RMMA index, RMMA-RAr index, and RMMA-nRAr index) separately for within-subjects factor (with versus without therapy) and to assess the interaction effect between the two factors (treatment effect of CPAP versus that of MAA). The baseline characteristics that showed significant differences between the two therapy groups were included as covariates of the two-way repeated measures ANOVA. Further, when CPAP and MAA showed significantly different effects on the RMMA variables, paired t-test or Wilcoxon signed-rank test were performed for each therapy group separately. The statistical analyses were performed in the total group as well as in individuals with comorbid SB. The significance level α of all statistical tests was set at 5%. All statistical analyses were performed using the SPSS statistics software package (version 26.0, SPSS Inc., Chicago, IL).

Results

Figure 1 shows a flow diagram of participants. We reviewed the medical history and PSG recordings of 2639 patients. According to the inclusion and exclusion criteria, 38 patients with OSA (32 men) were included in this study. Among them, 6 were diagnosed with mild OSA, 23 with moderate OSA, and 9 with severe OSA. Their median AHI was 23.7 events/hour (range from 8.6 to 75.9); their mean age was 52.6 ± 10.6 years; and their mean BMI was 25. 4 ± 6.3 kg/m2. Patients’ treatment duration varied from 75 to 598 days, with a median of 229 days. Thirteen individuals received CPAP and 25 received MAA therapy. In addition, 21 individuals (17 men) were diagnosed with comorbid SB (6 received CPAP and 15 received MAA).

Fig.1
figure 1

Flow diagram of participants in this study. PSG: polysomnography; OSA: obstructive sleep apnea; CPAP: continuous positive airway pressure; MAA: mandibular advancement appliance

Full size image

Sleep variables, including total sleep time, sleep efficiency, and percentages of sleep positions, did not change significantly with CPAP or MAA therapy (see Table 1). Both therapies reduced the AHI and ODI significantly (both P < 0.01). With regard to arousal-related variables, CPAP significantly decreased the total arousal index and RAr index (both P < 0.01), while MAA only reduced the RAr index (P = 0.006). Both therapies did not affect the nRAr index (P > 0.05).

Table 1 Characteristics and polysomnographic variables of two therapy groups
Full size table

The baseline characteristics and PSG variables of participants are presented in Table 1. None of the variables differed significantly between the CPAP and MAA therapy groups. Therefore, no variable was taken as a covariate in ANOVA. Table 2 shows the values of RMMA variables in the total group and in the subgroup (i.e., OSA individuals with comorbid SB). Both in the total group and in the subgroup, the RMMA index decreased significantly with OSA therapies (P < 0.05 and 0.01, respectively). In addition, based on the interaction effect, the changes in the RMMA index between PSG recordings without and with therapy did not differ significantly between CPAP and MAA (P > 0.05).

Table 2 Effects of OSA therapies on sleep bruxism a, b
Full size table

Figure 2 shows the individual values of RMMA indices in PSG recordings with and without therapy. Among individuals who received CPAP (n = 13), six (46%) showed a decrease in the RMMA index, while seven (54%) showed an increase. For MAA, the RMMA index decreased in 17 individuals (68%), while it increased in seven individuals (28%), and one case (4%) showed no difference in the RMMA index. In OSA individuals with comorbid SB, the RMMA index decreased in four of the six cases who received CPAP (67%) and in 12 of the 15 cases who received MAA therapy (80%).

Fig. 2
figure 2

Individual values of RMMA indices in polysomnographic recordings without and with CPAP and MAA. RMMA: rhythmic masticatory muscle activity; CPAP: continuous positive airway pressure (N = 13); MAA: mandibular advancement appliance (N = 25)

Full size image

In addition, in the total group, the decrease of the percentage of the RMMA index was (-25%|52%|82%). For CPAP and MAA, it was (-34%|-13%|61%) and (-2%|59%|86%), respectively. The changes of the percentage of the RMMA index showed no significant difference between CPAP and MAA (P = 0.159). In individuals with OSA and comorbid SB, the decrease of the percentage of the RMMA index was (16%|61%|84%). For CPAP and MAA, it was (-13%|48%|77%) and (47%|62%|88%), respectively. The changes of the percentage of the RMMA index showed no significant difference between CPAP and MAA (P = 0.381).

During CPAP and MAA therapy, the RMMA-RAr index reduced significantly, and no significant difference in the changes of the RMMA-RAr index was found between the two therapies. The RMMA-nRAr index showed no significant changes in the total group and in the subgroup. However, CPAP and MAA showed a significantly different effect on the changes in the RMMA-nRAr index (P < 0.05). For this, a post-hoc analysis was conducted to analyze the effect of CPAP and MAA separately. In the total group, the RMMA-nRAr index increased significantly with CPAP treatment (Z = -2.027, P = 0.043), while it did not change significantly with MAA in situ (Z = 1.786, P = 0.074). To the contrary, in individuals with OSA and SB, CPAP did not affect the RMMA-nRAr index (T = -1.644, P = 0.161), but MAA decreased the RMMA-nRAr index significantly (Z = -2.385, P = 0.017).

Discussion

Previous case reports showed that CPAP could relieve SB in cases with severe OSA, and several studies showed that MAA could do so in otherwise healthy individuals with SB [11,12,13, 21]. However, the effects of OSA therapies on SB has rarely been investigated in adults with OSA. Since SB could be primary, i.e., without clear cause, or secondary to other disorders [22], the underlying mechanism for the genesis of different phenotypes of SB might be different. Subsequently, the treatment effects of CPAP and MAA on SB may also vary depending on the characteristics of the population under investigation. Considering SB is highly prevalent in individuals with OSA (30% –50%) [10], it is clinically relevant to understand the effects of these therapies on SB in the OSA population.

As expected, both CPAP and MAA decreased the RMMA index in individuals with OSA as well as in those with comorbid SB. Therefore, we accepted our hypothesis for the primary aim that both CPAP and MAA will reduce the frequency of RMMA in individuals with OSA. These results are consistent with previous studies in individuals with primary SB [11, 13, 21]. Further, the results support that in individuals with comorbid OSA and SB, OSA therapies can improve both conditions [10]. Based on this, for individuals with concomitant OSA and SB, OSA therapies should be adopted in the first step. Then, sleep physicians should evaluate if the negative consequences of SB, such as jaw muscle pain, temporomandibular joint disorder or teeth wear, are still severe enough to necessitate a collaborative management by sleep physicians and dentists.

In addition, our results showed that there was no significant difference in the changes of the RMMA index between CPAP and MAA. This could be explained by several reasons. First, most of the participants in our sample were diagnosed with mild to moderate OSA. As reported in previous studies, CPAP and MAA did not show significant difference in the treatment of mild to moderate OSA [23, 24]. Thus, the possible difference between CPAP and MAA in the changes of RMMA that benefited from the improvement of OSA may not be present in adults with mild to moderate OSA. Further studies are recommended to confirm whether or not CPAP is more effective than MAA in relieving SB in individuals with severe OSA. Second, although SB may be a motor response to sleep arousals during sleep [25], some studies demonstrated that sleep arousal only acts as a permissive window for the occurrence of SB rather than a trigger [26]. Therefore, the reduction of sleep arousal may not have a linear relationship with that of RMMA. Based on this, although CPAP reduced more RAr than MAA (Table 1), it is possible that CPAP did not show a better effect on reducing the RMMA index than MAA. In addition, MAA has been reported to be able to relieve SB in individuals with SB but without OSA, implying that MAA may have other mechanisms for reducing RMMA compared with CPAP, e.g., by novelty effect of MAA, reducing the contractile properties of masseter muscles when the mandible is advanced by MAA, reducing mandibular movement freedom, and inducing masticatory muscle pain which then decreases RMMA [27].

Although both CPAP and MAA showed a consistent effect on the total RMMA index and RMMA-RAr index, the two therapies showed different effects on the RMMA-nRAr index. In the total group, the RMMA-nRAr index increased with CPAP therapy (P = 0.043), but no change was found with MAA in situ (P = 0.074). In individuals with OSA and SB, CPAP did not affect the RMMA-nRAr index (P = 0.161), while MAA decreased the RMMA-nRAr index significantly (P = 0.017). In short, MAA seems to have an additional effect on RMMA related to non-respiratory arousal compared with CPAP. This is consistent with the abovementioned notion that there are multiple mechanisms of MAA for reducing RMMA.

It is noteworthy that the effect of OSA therapy on SB seems to vary at an individual level. In the present study, around 60% of individuals with OSA (6/13 in CPAP, 17/25 in MAA) showed a decrease in the RMMA index with OSA treatment. In addition, the percentage of the changes of the RMMA index with OSA therapies among individuals had a wide range. This phenomenon has been reported in previous study [28]. There may be several reasons for such variances. First, previous study demonstrated that RMMA index has a time-variant nature across nights (22%-37%) [29]. Thus, the changes of the RMMA index in this study may be partially due to the natural fluctuation of RMMA. Second, although anterior tongue positions can reduce masticatory muscle activity and improve upper airway patency under CPAP and MAA, they may also increase masticatory muscle activity in some cases [30]. Third, the treatment durations of patients in this study presented a large variance, which may contribute to the individual differences, especially when considering the possible novelty effects of OSA therapies. Last but not least, according to the etiology of SB, SB could be primary or secondary related to other medical conditions [22]. Thus, we can speculate that RMMA episodes may have mixed pathogenesis both between and within individuals. Based on this, combined therapies, involving primary therapy for OSA and supplementary therapies for the remaining signs or symptoms of SB, would be better to improve patients’ quality of life.

Limitations

Several limitations should be taken into consideration when interpreting the results of this study. First, since this is not a randomized controlled trial (RCT), the comparison of the effects of CPAP and MAA on SB may be biased. Nevertheless, our results show that both CPAP and MAA can effectively relief SB in most cases. Future studies are recommended to adopt RCT design to compare the effects of OSA therapies, thus supplying solid evidence for treatment of cases with concomitant OSA and SB. Second, this study did not involve a healthy control group so that the observed effect of CPAP and MAA on SB cannot be definitively attributed to the therapies themselves. Third, a limited sample was included in this study, especially for the CPAP group. Besides, as shown in Fig. 2, two cases showed substantial decreases with CPAP and MAA therapies, which could be considered as outliers. However, possibly, these purported outliers may be common observations in a large sample study. This could be supported by the large interindividual discrepancy in the effects of CPAP and MAA on SB that was presented in this study and by previous case reports which showed that SB episodes could disappear completely with CPAP treatment [11, 12]. For that reason, we did not remove these cases from analysis. Fourth, the signs and symptoms of SB before and after treatment, the actual treatment durations, patients’ adherence, and the decision-making process of OSA therapy for each patient, were not collected at the moment of data collection. Future RCTs with large samples are needed to confirm our findings in individuals with OSA and to compare the effects of different treatment modalities with different configurations.

Conclusions

Within the limitations of this study, we concluded that both continuous positive airway pressure and mandibular advancement appliance significantly reduce sleep bruxism in individuals with obstructive sleep apnea and in those with comorbid sleep bruxism. No significant difference regarding the effects on sleep bruxism was found between the two therapies. In addition, the two therapies can reduce sleep bruxism episodes related to respiratory arousal but may have different effects on those related to non-respiratory arousal. It is noteworthy that the interindividual differences in the effects of both therapies on sleep bruxism are large.