1 Introduction

Variable Lebesgue spaces \(L_{p(\cdot )}\) are investigated very intensively in the literature nowadays (see e.g. Cruz-Uribe and Fiorenza [5], Diening et al. [8], Kokilashvili et al. [23, 24], Kováčik and Rákosník [25], Cruz-Uribe et al. [3], Nakai and Sawano [29, 37], Jiao et al. [16, 19, 21], Yan et al. [50], Liu et al. [26, 27]). Interest in variable Lebesgue spaces has increased since the 1990s because of their use in a variety of applications (see the references in Jiao et al. [16]).

In this theory, usually, we suppose that \(p(\cdot )\) or \(1/p(\cdot )\) satisfies the log-Hölder continuity condition. One of the most important results states that the classical Hardy-Littlewood maximal operator, M is bounded on the variable \(L_{p(\cdot )}\) spaces under this condition and if \(1 < p_{-} \le p_{+} \le \infty \), where \(p_{-}\) denotes the infimum and \(p_{+}\) the supremum of \(p(\cdot )\) (see for example Cruz-Uribe et.al [3], Nekvinda [30], Cruz-Uribe and Fiorenza [5] and Diening et al. [6, 8]). The fractional maximal operator \(M_{\alpha }\) generates the Hardy-Littlewood operator and is bounded from \(L_{q(\cdot )}\) to \(L_{p(\cdot )}\), whenever \(0 \le \alpha <1\), \(1<q_-\le q_+ \le 1/\alpha \) and \(1/p(\cdot )= 1/q(\cdot )- \alpha \) (see Capone et al. [2], Cruz-Uribe and Fiorenza [5] or Kokilashvili and Meskhi [22]). The fractional integral operator was investigated in Diening [7], Ephremidze et al. [9], Mizuta and Shimomura [28], Rafeiro and Samko [35], Izuki et al. [15] and, for martingales, in Arai et al. [1], Hao et al. [13, 20], Jiao et al. [17] and Sadasue [36].

The analogous boundedness result for the martingale maximal operator on the \(L_{p(\cdot )}\) spaces was proved in [16, 19] if \(1<p_-\le p_+<\infty \). Later we generalized this result to \(1<p_-\le p_+ \le \infty \) in [45]. In [16, 43, 44, 46], we investigated more general maximal operators for dyadic martingales and verified that they are bounded on \(L_{p(\cdot )}\) if \(1<p_-\le p_+<\infty \). These operators were the key points in the proof of the boundedness of the maximal Fejér operator of the Walsh-Fourier series from the variable Hardy space \(H_{p(\cdot )}\) to \(L_{p(\cdot )}\) (see [16, 40, 46]). In this paper we generalize these operators and introduce the general fractional maximal operator for more general martingales.

Nakai and Sawano [29] first introduced the variable Hardy spaces \(H_{p(\cdot )}({\mathbb {R}})\). Independently, Cruz-Uribe and Wang [4] also investigated the spaces \(H_{p(\cdot )}({\mathbb {R}})\). Sawano [37] improved the results in [29]. Ho [14] studied weighted Hardy spaces with variable exponents. Yan et al. [50] introduced the variable weak Hardy spaces \(H_{p(\cdot ),\infty }({\mathbb {R}})\) and characterized these spaces via radial maximal functions. The Hardy-Lorentz spaces \(H_{p(\cdot ),q}({\mathbb {R}})\) were investigated by Jiao et al. in [21]. Similar results for the anisotropic Hardy spaces \(H_{p(\cdot )}({\mathbb {R}})\) and \(H_{p(\cdot ),q}({\mathbb {R}})\) can be found in Liu et al. [26, 27]. The theory of martingale Hardy spaces with variable exponents was started with the paper Jiao et al. [19]. Some years later, we have systematically studied and developed this theory in [16]. The applications of variable martingale Hardy spaces to Fourier analysis were investigated in [16] while its applications to stochastic integrals in [18]. Martingale Musielak-Orlicz Hardy spaces were considered in Xie et al. [47,48,49].

In this paper, we generalize all the maximal operators mentioned above and introduce a new, general fractional maximal operator \(M_{\gamma ,s,\alpha }\) for so called Vilenkin martingales \((\gamma ,s>0, 0<\alpha \le 1)\). This maximal operator is larger than the Doob maximal operator or the original fractional maximal operator. We generalize the boundedness of the Doob and fractional maximal operators and prove that \(M_{\gamma ,s,\alpha }\) is bounded from \(L_{q(\cdot )}\) to \(L_{p(\cdot )}\) if \(1/p(\cdot )\) is log-Hölder continuous, \(1<q_-\le q_+ \le 1/\alpha \), \(1/p(\cdot )= 1/q(\cdot )- \alpha \) and \(1/p_- - 1/p_+ < \gamma +s\). Under the same conditions, we obtain also the boundedness of \(M_{\gamma ,s,\alpha }\) from \(H_{q(\cdot )}\) to \(L_{p(\cdot )}\). Moreover, we show that for \(\alpha =0\), \(\Vert M_{\gamma ,s,0}f\Vert _{L_{p(\cdot )}}\) is equivalent to \(\Vert f\Vert _{H_{p(\cdot )}}\), where \(H_{p(\cdot )}\) denotes the variable martingale Hardy spaces. Finally, we show that the condition \(\frac{1}{p_-}-\frac{1}{p_+} < \gamma +s\) is important.

2 Variable Lebesgue spaces

In this section, we recall some basic notations on variable Lebesgue spaces and give some elementary and necessary facts about these spaces. Our main references are Cruz-Uribe and Fiorenza [5] and Diening et al. [8].

For a constant p, the \(L_p\) space is equipped with the (quasi)-norm

$$\begin{aligned} \Vert f\Vert _p:=\left( \int _{0}^{1}|f(x)|^p \, dx\right) ^{1/p} \qquad (0<p<\infty ), \end{aligned}$$

with the usual modification for \(p=\infty \). Here we integrate with respect to the Lebesgue measure \(\lambda \). For the integral of f, we will use both symbols \(\int _{0}^{1} f(x) \, dx\) and \(\int _{0}^{1} f \, d \lambda \).

We are going to generalize these spaces. A measurable function \(p(\cdot ): [0,1)\rightarrow (0,\infty ]\) is called a variable exponent. For a measurable set \(A\subset [0,1)\), we denote

$$\begin{aligned} p_-(A):=\text {ess} \inf \limits _{x\in A}p(x),\quad p_+(A):=\text {ess} \sup \limits _{x\in A}p(x) \end{aligned}$$

and for convenience

$$\begin{aligned} p_-:=p_-( [0,1)),\quad p_+:=p_+( [0,1)). \end{aligned}$$

Denote by \({\mathcal {P}}\) the collection of all variable exponents \(p(\cdot )\) such that

$$\begin{aligned} 0<p_-\le p_+ \le \infty . \end{aligned}$$

To introduce the variable Lebesgue spaces, let

$$\begin{aligned} \rho (f):=\int _{ [0,1) \setminus \varOmega _\infty } |f(x)|^{p(x)} dx + \Vert f\Vert _{L_\infty (\varOmega _\infty )}, \end{aligned}$$

where \(\varOmega _\infty =\{x \in [0,1): p(x)=\infty \}\). We denote the set \([0,1) \setminus \varOmega _\infty \) also by \(\varOmega _\infty ^{c}\). The variable Lebesgue space \(L_{p(\cdot )}\) is the collection of all measurable functions f for which there exists \(\nu >0\) such that

$$\begin{aligned} \rho \left( {f}/{\nu }\right) <\infty . \end{aligned}$$

We equip \(L_{p(\cdot )}\) with the (quasi)-norm

$$\begin{aligned} \Vert f\Vert _{p(\cdot )}:=\inf \{\nu >0:\rho ({f}/{\nu })\le 1\}. \end{aligned}$$

If \(p(\cdot )=p\) is a constant, then we get back the definition of the usual Lebesgue spaces \(L_p\). For any \(f\in L_{p(\cdot )}\), we have \(\rho (f)\le 1\) if and only if \(\Vert f\Vert _{p(\cdot )}\le 1\). It is known that \(\Vert \nu f\Vert _{p(\cdot )}=|\nu |\Vert f\Vert _{p(\cdot )}\) and

$$\begin{aligned} \left\| |f|^s \right\| _{p(\cdot )}=\Vert f\Vert _{s p(\cdot )}^s, \end{aligned}$$

where \(p(\cdot )\in {\mathcal {P}}\), \(s\in (0,\infty )\), \(\nu \in {{\mathbb {C}}}\) and \(f\in L_{p(\cdot )}\). Details can be found in the monographs Cruz-Uribe and Fiorenza [5] and Diening et al. [8]. The variable exponent \(p'(\cdot )\) is defined pointwise by

$$\begin{aligned} \frac{1}{p(x)}+\frac{1}{p'(x)}=1, \quad x\in [0,1). \end{aligned}$$

The next lemma is well known, see Cruz-Uribe and Fiorenza [5] or Diening et al. [8].

Lemma 1

Let \(p(\cdot ) \in {\mathcal {P}}\) with \(1\le p_- \le p_+ \le \infty \). For all \(f \in L_{p(\cdot )}\) and \(g \in L_{p'(\cdot )}\),

$$\begin{aligned} \int _0^{1} \left| fg\right| \, d\lambda \le C_{p(\cdot )} \left\| f\right\| _{p(\cdot )} \left\| g\right\| _{p'(\cdot )}. \end{aligned}$$

Moreover,

$$\begin{aligned} \Vert f\Vert _{p(\cdot )}\sim \sup _{\Vert g\Vert _{p'(\cdot )}\le 1}\left| \int _{0}^{1} fg \, d \lambda \right| , \end{aligned}$$

where \(\sim \) denotes the equivalence of the numbers.

We denote by \(C^{\log }\) the set of all functions \(p(\cdot )\in {\mathcal {P}}\) satisfying the so-called log-Hölder continuous condition, namely, there exists a positive constant \(C_{\log }(p)\) such that, for any \(x,y\in [0,1)\),

$$\begin{aligned} |p(x)-p(y)|\le \frac{C_{\log }(p)}{\log (e+1/|x-y|)}. \end{aligned}$$
(2.1)

Remark 1

There exist a lot of functions \(p(\cdot )\) satisfying (2.1). For concrete examples we mention the function \(a+cx\) for parameters a and c such that the function is positive (\(x \in [0,1)\)). All positive Lipschitz functions with order \(0< \eta \le 1\) also satisfy (2.1). Under the condition \(0<p_-\le p_+<\infty \), \(p(\cdot ) \in C^{\log }\) if and only if \(1/p(\cdot ) \in C^{\log }\).

The following two lemmas were proved in Cruze-Uribe and Fiorenza [5] (see also Hao and Jiao [13]).

Lemma 2

If \(1/p(\cdot ) \in C^{\log }\), then there exists a constant \(0<\beta <1\) such that for all intervals \(I \subset [0,1)\),

$$\begin{aligned} \lambda (I)^{1/p_+(I)-1/p_-(I)} \le \frac{1}{\beta }. \end{aligned}$$
(2.2)

Lemma 3

If \(1/p(\cdot ) \in C^{\log }\), then for any interval \(I \subset [0,1)\) and \(x\in I\),

$$\begin{aligned} \lambda (I)^{1/{p_-(I)}} \sim \lambda (I)^{1/{p(x)}} \sim \lambda (I)^{1/{p_+(I)}} \sim \Vert \chi _I\Vert _{p(\cdot )}. \end{aligned}$$

For general martingale Hardy spaces, instead of the log-Hölder continuity condition, we supposed in [13, 16, 20, 40, 45] the slightly more general condition (2.2) for all atoms of the \(\sigma \)-algebras.

In this paper the constants C are absolute constants and the constants \(C_{p(\cdot )}\) are depending only on \(p(\cdot )\) and may denote different constants in different contexts. For two positive numbers A and B, we use also the notation \(A \lesssim B\), which means that there exists a constant C such that \(A \le CB\).

3 Variable martingale Hardy spaces

Let \((p_n, n \in {{\mathbb {N}}})\) be a sequence of natural numbers with entries at least 2. We always suppose that the sequence \((p_n)\) is bounded. Let

$$\begin{aligned} {\widehat{p}}:= \sup _{n \in {\mathbb {N}}}p_n<\infty . \end{aligned}$$
(3.1)

Introduce the notations \(P_0=1\) and

$$\begin{aligned} P_{n+1}:= \prod _{k=0}^n p_k \qquad (n \in {{\mathbb {N}}}). \end{aligned}$$

Let \({{\mathcal {F}}}_n\) be the \(\sigma \)-algebra

$$\begin{aligned} {{\mathcal {F}}}_{n} = \sigma \{ [kP_n^{-1},(k+1)P_n^{-1}) \, :0 \le k <P_n \}, \end{aligned}$$

where \(\sigma ({{{\mathcal {H}}}})\) denotes the \(\sigma \)-algebra generated by an arbitrary set system \({\mathcal {H}}\). By a Vilenkin interval we mean one of the form \([kP_n^{-1},(k+1)P_n^{-1})\) for some \(k,n \in {{\mathbb {N}}}\), \(0\le k <P_n\). The conditional expectation operators relative to \(\mathcal{F}_n\) are denoted by \(E_n\). An integrable sequence \(f = \left( f_{n}\right) _{n \in {{\mathbb {N}}}}\) is said to be a martingale if \(f_{n}\) is \({\mathcal {F}}_{n}\)-measurable for all \(n \in {{\mathbb {N}}}\) and \(E_{n} f_{m} = f_{n}\) in case \(n \le m\). Martingales with respect to \(({\mathcal {F}}_n, n \in {{\mathbb {N}}})\) are called Vilenkin martingales. Vilenkin martingales were introduced in a great number of papers, such as Gát and Goginava [10,11,12], Persson and Tephnadze [31,32,33,34] and Simon [38, 39]. It is easy to show (see e.g. Weisz [41]) that the sequence \(({\mathcal {F}}_n, n \in {{\mathbb {N}}})\) is regular, i.e., \(f_n\le {\widehat{p}} f_{n-1}\) for all non-negative Vilenkin martingales with \({\widehat{p}}\) just defined in (3.1).

For a Vilenkin martingale \(f = \left( f_{n}\right) _{n \in {{\mathbb {N}}}}\), the Doob maximal function is defined by

$$\begin{aligned} M(f) := \sup _{n \in {{\mathbb {N}}}} \left| f_{n} \right| . \end{aligned}$$

For a fixed \(x \in [0,1)\) and \(n \in {\mathbb {N}}\), let us denote the unique Vilenkin interval \([kP_n^{-1},(k+1)P_n^{-1})\) which contains x by \(I_n(x)\). Then it is easy to see that

$$\begin{aligned} M(f)(x)= \sup _{n \in {\mathbb {N}}} \frac{1}{\lambda (I_n(x))}\left| \int _{I_n(x)} f_n d \lambda \right| . \end{aligned}$$
(3.2)

If \(f\in L_1\), then we can replace \(f_n\) by f in the integral. Now we can define the variable martingale Hardy spaces by

$$\begin{aligned} H_{p(\cdot )}:= & {} \left\{ f = \left( f_{n}\right) _{n \in {{\mathbb {N}}}}: \left\| f\right\| _{H_{p(\cdot )}} := \left\| M(f)\right\| _{p(\cdot )} < \infty \right\} . \end{aligned}$$

The Hardy spaces can also be defined via equivalent norms. For a martingale \(f=(f_n)_{n\ge 0}\), let

$$\begin{aligned} d_{n}f=f_{n}-f_{n-1} \qquad (n\ge 0) \end{aligned}$$

denote the martingale differences, where \(f_{-1}:=0\). The square function and the conditional square function of f are defined by

$$\begin{aligned} S(f)=\left( \sum _{n=0}^{\infty }|d_nf|^2\right) ^{1/2}, \qquad s(f)=\left( |d_0f|^{2}+\sum \limits _{n=0}^{\infty }E_{n}|d_{n+1} f|^2\right) ^{1/2}. \end{aligned}$$

We have shown the following theorem in [16].

Theorem 1

Let \(p(\cdot ) \in C^{\log }\) and \(0< p_-\le p_+ < \infty \). Then

$$\begin{aligned} \left\| M(f)\right\| _{p(\cdot )} \sim \left\| S(f)\right\| _{p(\cdot )} \sim \left\| s(f)\right\| _{p(\cdot )}. \end{aligned}$$

If in addition \(1< p_-\le p_+ < \infty \), then \(H_{p(\cdot )} \sim L_{p(\cdot )}\).

In this paper, we will give more equivalent characterizations for the variable Hardy spaces using the new maximal functions.

The atomic decomposition is a useful characterization of the Hardy spaces. A measurable function a is called a \(p(\cdot )\)-atom if there exists a Vilenkin interval I such that

  1. (a)

    the support of a is contained in I,

  2. (b)

    \(\Vert M(a)\Vert _{\infty } \le \left\| \chi _{I} \right\| _{p(\cdot )}^{-1}\),

  3. (c)

    \(\int _{I}a \, d \lambda =0\).

The atomic decompositions of the spaces \(H_{p(\cdot )}\) were proved in Jiao et al. [16, 19]. The classical case can be found in [41, 42].

Theorem 2

Let \(p(\cdot ) \in C^{\log }\) and \(0< p_-\le p_+ < \infty \). Then the martingale \(f = \left( f_{n}\right) _{n \in {{\mathbb {N}}}} \in H_{p(\cdot )}\) if and only if

$$\begin{aligned} f_{n} = \sum _{k \in {\mathbb {Z}}} \sum _{K\in {{\mathbb {N}}}} \sum _{\kappa } \mu _{k,K,\kappa } E_na^{k,K,\kappa } \qquad \text{ almost } \text{ everywhere } \end{aligned}$$

for every \(n \in {{\mathbb {N}}}\), where \((a^{k,K,\kappa })_{k\in {\mathbb {Z}}, K,\kappa \in {\mathbb {N}}}\) is a sequence of \(p(\cdot )\)-atoms associated with the Vilenkin intervals \((I_{k,K,\kappa })_{k\in {\mathbb {Z}}, K,\kappa \in {\mathbb {N}}}\subset {\mathcal {F}}_K\), which are disjoint for fixed k, and \(\mu _{k,K,\kappa }=3\cdot 2^k \Vert \chi _{I_{k,K,\kappa }}\Vert _{p(\cdot )}\). Moreover,

$$\begin{aligned} \left\| \left( \sum _{k \in {{\mathbb {Z}}}} \sum _{K\in {{\mathbb {N}}}} \sum _{\kappa } \left( \frac{\mu _{k,K,\kappa } \chi _{ I_{k,K,\kappa }}}{ \left\| \chi _{ I_{k,K,\kappa }} \right\| _{p(\cdot )}} \right) ^{t} \right) ^{1/t} \right\| _{p(\cdot )} \lesssim \left\| f \right\| _{H_{p(\cdot )}}, \end{aligned}$$

where \(0 < t \le {\underline{p}}:=\min \{p_-,1\}\) is fixed and the infimum is taken over all decompositions of f as above.

4 The boundedness of fractional maximal operators on \(L_{p(\cdot )}\)

Let \(0 \le \alpha \le 1\). The original fractional maximal operator

$$\begin{aligned} M_{\alpha }(f)(x)= \sup _{n \in {\mathbb {N}}} \frac{1}{\lambda (I_n(x))^{1-\alpha }} \left| \int _{I_n(x)} f_n d \lambda \right| \end{aligned}$$

generates the Doob maximal operator M. Indeed, for \(\alpha =0\), we get back the definition of M because of (3.2). If f is an integrable function, then we can replace \(f_n\) by f in the definition of \(M_{\alpha }\).

The following result was proved for the classical fractional maximal operator and for \(L_{p(\cdot )}({\mathbb {R}})\) spaces in Capone [2], Cruz-Uribe and Fiorenza [5] or Kokilashvili and Meskhi [22] and for more general martingales in Jiao et al. [16, 19] if \(p_+<\infty , \alpha =0\) and by the author [45] if \(p_+ \le \infty , \alpha =0\).

Theorem 3

Let \(1/q(\cdot ) \in C^{\log }\), \(0 \le \alpha <1\) and \(1<q_-\le q_+ \le 1/\alpha \). If

$$\begin{aligned} \frac{1}{p(\cdot )}= \frac{1}{q(\cdot )}- \alpha , \end{aligned}$$

then

$$\begin{aligned} \Vert M_{\alpha }(f)\Vert _{p(\cdot )} \lesssim \Vert f\Vert _{{q(\cdot )}} \qquad (f\in L_{q(\cdot )}). \end{aligned}$$

We will generalize the preceding martingale fractional maximal operator. Every point \(x \in [0,1)\) can be written in the following way:

$$\begin{aligned} x= \sum _{k=0}^{\infty } \frac{x_k}{P_{k+1}} \qquad (0 \le x_k<p_k, \; x_k \in {{{\mathbb {N}}}}). \end{aligned}$$

If there are two different forms, choose the one for which \(\lim _{k \rightarrow \infty } x_k =0\). The so called Vilenkin addition is defined by

$$\begin{aligned} x\dot{+} y = \sum _{k=0}^{\infty } \frac{z_k}{P_{k+1}}, \qquad \text{ where } z_k:= x_k+y_k \ \text{ mod } \ p_k, (k \in {{{\mathbb {N}}}}). \end{aligned}$$

Given two Vilenkin intervals I and J, let \(I \dot{+}J:=\{x\dot{+}y: x \in I, y \in J\}\). For a Vilenkin interval I with length \(P_n^{-1}\), \(i,j,n \in {\mathbb {N}}\), \(l=0,\ldots ,p_j-1\), let us use the notation

$$\begin{aligned} I^{l,j,i}:= I \dot{+} [0,P_{i}^{-1}) \dot{+} l P_{j+1}^{-1}. \end{aligned}$$

Parallel, we denote \(I_n(x)^{l,j,i}:= (I_n(x))^{l,j,i}\). Recall that \(I_n(x)\) is a Vilenkin interval such that \(I_n(x) \in {\mathcal {F}}_n\) and \(x \in I_n(x)\). Let \(\gamma \) and s be two positive constants. Now we introduce four new fractional maximal functions and a common generalization of these maximal functions. For a martingale \(f=(f_n)_{n \ge 0}\), let

$$\begin{aligned} M_{\gamma ,s,\alpha }^{(1)}(f)(x)&:= \sup _{n \in {\mathbb {N}}} \sum _{l=0}^{p_n-1} \frac{1}{\lambda (I_n(x)^{l,n,n})^{1-\alpha }}\left| \int _{I_n(x)^{l,n,n}} f_n d \lambda \right| \\&= \sup _{n \in {\mathbb {N}}} \frac{p_n}{\lambda (I_n(x))^{1-\alpha }}\left| \int _{I_n(x)} f_n d \lambda \right| . \end{aligned}$$

Note that \(I_n(x)^{l,n,n}= I_n(x)\) (\(n \in {\mathbb {N}}\), \(l=0,\ldots ,p_n-1\)). Let

$$\begin{aligned}&M_{\gamma ,s,\alpha }^{(2)}(f)(x) \\&:= \sup _{n \in {\mathbb {N}}} \sum _{m=0}^{n} \left( \frac{P_m}{P_n} \right) ^{\gamma } \sum _{l=0}^{p_m-1} \frac{1}{\lambda (I_n(x)^{l,m,m})^{1-\alpha }}\left| \int _{I_n(x)^{l,m,m}} f_n d \lambda \right| \\&= \sup _{n \in {\mathbb {N}}} \sum _{m=0}^{n} \left( \frac{P_m}{P_n} \right) ^{\gamma } \frac{p_m}{\lambda (I_m(x))^{1-\alpha }}\left| \int _{I_m(x)} f_n d \lambda \right| . \end{aligned}$$

Here \(I_n(x)^{l,m,m} = I_n(x)\dot{+}[0,P_m^{-1}) \dot{+} l P_{m+1}^{-1}=x\dot{+}[0,P_m^{-1})=I_m(x)\). Note that \(M_{\gamma ,s,\alpha }^{(1)}\) and \(M_{\gamma ,s,\alpha }^{(2)}\) are equivalent to \(M_{\alpha }\), more exactly,

$$\begin{aligned} M_{\alpha }(f) \le M_{\gamma ,s,\alpha }^{(1)}f \le M_{\gamma ,s,\alpha }^{(2)}f \le C M_{\alpha }(f) \end{aligned}$$

and so Theorem 3 holds also for these two operators.

The third fractional maximal operator is defined by

$$\begin{aligned} M_{\gamma ,s,\alpha }^{(3)}(f)(x) := \sup _{n \in {\mathbb {N}}} \sum _{j=0}^{n} \left( \frac{P_j}{P_n} \right) ^{\gamma +s} \sum _{l=0}^{p_j-1} \frac{1}{\lambda (I_n(x)^{l,j,n})^{1-\alpha }}\left| \int _{I_n(x)^{l,j,n}} f_n d \lambda \right| . \end{aligned}$$

Note that \(I_n(x)^{l,j,n}= I_n(x) \dot{+} [0,P_{n}^{-1}) \dot{+} l P_{j+1}^{-1}= I_n(x) \dot{+} l P_{j+1}^{-1}\).

We define our last fractional maximal operator by

$$\begin{aligned}&M_{\gamma ,s,\alpha }^{(4)}(f)(x) \\&:= \sup _{n \in {\mathbb {N}}} \sum _{j=0}^{n} \left( \frac{P_j}{P_n} \right) ^{\gamma } \sum _{i=j}^{n} \left( \frac{P_j}{P_i} \right) ^{s} \sum _{l=0}^{p_j-1}\frac{1}{\lambda (I_n(x)^{l,j,i})^{1-\alpha }}\left| \int _{I_n(x)^{l,j,i}} f_n d \lambda \right| . \end{aligned}$$

The maximal operators \(M_{\gamma ,s,\alpha }^{(3)}\) and \(M_{\gamma ,s,\alpha }^{(4)}\) cannot be estimated by \(M_{\alpha }\) from above pointwise. For \(\alpha =0\), we considered these two maximal operators in [46] and used them to verify some boundedness and convergence results for the Fejér means of the Vilenkin-Fourier series. We proved there that they are bounded on \(L_{p(\cdot )}\) if \(1/p(\cdot ) \in C^{\log }\) and \(1< p_-\le p_+<\infty \). In this paper, we will generalize this result for a more general operator, for \(1< p_-\le p_+ \le \infty \) and \(0<\alpha \le 1\). The next general fractional maximal operator generalizes our operators \(M_{\gamma ,s,\alpha }^{(j)}\) \((j=1,\ldots ,4)\). Let

$$\begin{aligned}&M_{\gamma ,s,\alpha }(f)(x) \\&:= \sup _{n \in {\mathbb {N}}} \sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma } \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{s} \sum _{l=0}^{p_j-1}\frac{1}{\lambda (I_n(x)^{l,j,i})^{1-\alpha }}\left| \int _{I_n(x)^{l,j,i}} f_n d \lambda \right| . \end{aligned}$$

Of course, if \(f\in L_1\), then we can write again in the definition f instead of \(f_n\). For \(\alpha =0\), we omit simply \(\alpha \) and write \(M_{\gamma ,s}(f)\) and M(f). Obviously, we obtain from \(M_{\gamma ,s,\alpha }(f)\) the operator \(M_{\gamma ,s,\alpha }^{(1)}f\) if \(j=i=n=m\), \(M_{\gamma ,s,\alpha }^{(2)}f\) if \(j=i=m\), \(M_{\gamma ,s,\alpha }^{(3)}f\) if \(m=n\) and \(i=n\), \(M_{\gamma ,s,\alpha }^{(4)}f\) if \(m=n\), respectively.

It is easy to see that

$$\begin{aligned} M_{\alpha }(f) \le M_{\gamma ,s,\alpha }^{(j)}f \le M_{\gamma ,s,\alpha }(f) \qquad (j=1,\ldots ,4). \end{aligned}$$
(4.1)

Let us define \(I_{k,n}:= [kP_n^{-1},(k+1)P_n^{-1})\), where \(0\le k<P_n\), \(n\in {{\mathbb {N}}}\). The definition of \(M_{\gamma ,s,\alpha }(f)\) can be rewritten to

$$\begin{aligned}&M_{\gamma ,s,\alpha }(f) \\&:= \sup _{n\in {{\mathbb {N}}}} \sum _{k=0}^{P_n-1} \chi _{I_{k,n}}\sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma } \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{s} \sum _{l=0}^{p_j-1} \frac{1}{\lambda (I^{l,j,i}_{k,n})^{1-\alpha }}\left| \int _{I^{l,j,i}_{k,n}} f_n d \lambda \right| , \end{aligned}$$

where \(I^{l,j,i}_{k,n}:=(I_{k,n})^{l,j,i}\).

In this section, we will use the next lemma proved in [45]. A first version of this lemma can be found in Jiao et al. [16, 19].

Lemma 4

Let \(1/p(\cdot ) \in C^{\log }\) and \(1\le p_-\le p_+ \le \infty \). Suppose that \(f\in L_{p(\cdot )}\) with \(\left\| f\right\| _{p(\cdot )}\le 1\), \(f=f \chi _{\{|f| > 1\}}\) and \(\mathrm{supp \, }f \subset \varOmega _\infty ^{c}\). Then, for any interval \(I \subset [0,1)\) with \(\lambda (I \cap \varOmega _\infty ^{c})>0\) and for any \(p_-(I) \le r \le p_+(I)\) \((r<\infty )\),

$$\begin{aligned} \left( \frac{\beta }{\lambda (I)} \int _I |f(y)| \, dy\right) ^{r} \le \frac{1}{\lambda (I)} \int _I |f(y)|^{p(y)}\, dy \end{aligned}$$

where \(\beta \) is defined in (2.2).

We will also use the following simple lemma.

Lemma 5

Suppose that \(0 \le \alpha <1\) and \(1 \le r \le 1/\alpha \). Then for all intervals \(I \subset [0,1)\) and all functions \(f \in L_r\) with \(\Vert f\Vert _r \le 1\),

$$\begin{aligned} \frac{1}{\lambda (I)^{1- \alpha }} \int _{I} |f| \,d \lambda \le \left( \frac{1}{\lambda (I)} \int _{I} |f| \,d \lambda \right) ^{1- \alpha r}. \end{aligned}$$

Proof

We have

$$\begin{aligned} \frac{1}{\lambda (I)^{1- \alpha }} \int _{I} |f| \,d \lambda&= \lambda (I)^\alpha \left( \frac{1}{\lambda (I)} \int _{I} |f| \,d \lambda \right) ^{\alpha r} \left( \frac{1}{\lambda (I)} \int _{I} |f| \,d \lambda \right) ^{1- \alpha r}. \end{aligned}$$

The first term is bounded by 1 because, by Hölder’s inequality,

$$\begin{aligned} \lambda (I)^\alpha \left( \frac{1}{\lambda (I)} \int _{I} |f| \,d \lambda \right) ^{\alpha r} \le \lambda (I)^{\alpha - \alpha r} \lambda (I)^{\alpha r/r'} \Vert f\Vert _r^{\alpha r} \le 1, \end{aligned}$$

which finishes the proof of the lemma. \(\square \)

Our first boundedness result on the general fractional operator with constant p and q can be read as follows.

Theorem 4

Let \(0 \le \alpha <1\), \(1<q\le 1/\alpha \) and \(0<\gamma ,s<\infty \). If

$$\begin{aligned} \frac{1}{p}= \frac{1}{q}- \alpha , \end{aligned}$$

then

$$\begin{aligned} \Vert M_{\gamma ,s,\alpha }(f)\Vert _{p} \lesssim \Vert f\Vert _{q} \qquad (f\in L_{q}). \end{aligned}$$

Proof

For \(\alpha =0\), the theorem can be proved similarly to Corollary 4 in [46], so we omit the details. For \(0<\alpha <1\), we first note that

$$\begin{aligned} \sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma } \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{s} \sum _{l=0}^{p_j-1}1&\le \sum _{m=0}^{n} \sum _{j=0}^{m} 2^{(j-n)\gamma } \sum _{i=j}^{m} 2^{(j-i)s} p_j \nonumber \\&\le \frac{2^{s}}{2^{s}-1} \frac{2^{2 \gamma }}{(2^{\gamma }-1)^{2}}{\widehat{p}} := C_s C_\gamma {\widehat{p}}. \end{aligned}$$
(4.2)

Since \(p \ge q\), we get by convexity that

$$\begin{aligned} M_{\gamma ,s,\alpha }(f)^{p/q}&= \sup _{n\in {{\mathbb {N}}}} \sum _{k=0}^{P_n-1} \chi _{I_{k,n}} \Bigg (\sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma } \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{s} \\&\qquad \sum _{l=0}^{p_j-1} \frac{1}{C_s C_\gamma {\widehat{p}}}\frac{C_s C_\gamma {\widehat{p}}}{\lambda (I^{l,j,i}_{k,n})^{1-\alpha }}\left| \int _{I^{l,j,i}_{k,n}} f d \lambda \right| \Bigg )^{p/q} \\&\le \sup _{n\in {{\mathbb {N}}}} \sum _{k=0}^{P_n-1} \chi _{I_{k,n}} \sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma } \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{s} \\&\qquad \sum _{l=0}^{p_j-1} \frac{1}{C_s C_\gamma {\widehat{p}}} \left( \frac{C_s C_\gamma {\widehat{p}}}{\lambda (I^{l,j,i}_{k,n})^{1-\alpha }}\left| \int _{I^{l,j,i}_{k,n}} f d \lambda \right| \right) ^{p/q}. \end{aligned}$$

Observe that \(p(1- \alpha q)/q=1\). Now we can apply Lemma 5 to obtain

$$\begin{aligned} M_{\gamma ,s,\alpha }(f)&\lesssim \Bigg (\sup _{n\in {{\mathbb {N}}}} \sum _{k=0}^{P_n-1} \chi _{I_{k,n}} \sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma } \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{s} \sum _{l=0}^{p_j-1} \\&\qquad \left( \frac{1}{\lambda (I^{l,j,i}_{k,n})}\left| \int _{I^{l,j,i}_{k,n}} f d \lambda \right| \right) ^{p(1- \alpha q)/q}\Bigg )^{q/p}\\&= M_{\gamma ,s,0}(f)^{q/p}. \end{aligned}$$

Hence

$$\begin{aligned} \int _{0}^{1} \left| M_{\gamma ,s,\alpha }(f)\right| ^{p} \, d \lambda \lesssim \int _{0}^{1} \left| M_{\gamma ,s,0}(f)\right| ^{q} \, d \lambda \lesssim \int _{0}^{1} \left| f\right| ^{q}\, d \lambda , \end{aligned}$$

which finishes the proof. \(\square \)

Now we generalize this theorem as well as Theorem 3 to variable Lebesgue spaces as follows. The methods used in the proof of Theorems 3 and 4 do not work now, so we need new ideas.

Theorem 5

Let \(1/q(\cdot ) \in C^{\log }\), \(0 \le \alpha <1\), \(1<q_-\le q_+ \le 1/\alpha \) and \(0<\gamma ,s<\infty \). If

$$\begin{aligned} \frac{1}{p(\cdot )}= \frac{1}{q(\cdot )}- \alpha , \end{aligned}$$
(4.3)

and

$$\begin{aligned} \frac{1}{p_-}-\frac{1}{p_+} < \gamma +s, \end{aligned}$$
(4.4)

then

$$\begin{aligned} \Vert M_{\gamma ,s,\alpha }(f)\Vert _{p(\cdot )} \lesssim \Vert f\Vert _{{q(\cdot )}} \qquad (f\in L_{q(\cdot )}). \end{aligned}$$

Proof

It is enough to show the theorem for \(\left\| f\right\| _{q(\cdot )} = 1\) and for non-negative functions. We decompose f as \(f^{1}+f^{2}\), where

$$\begin{aligned} f^{1}=f \chi _{\{f > 1\}}, \qquad f^{2}=f \chi _{\{f \le 1\}}. \end{aligned}$$

Then \(\left\| f^{i}\right\| _{q(\cdot )} \le 1\) and \(\rho (f^{i}) \le 1\), \(i=1,2\). Similarly to (4.2),

$$\begin{aligned} M_{\gamma ,s,\alpha }(f^{2})&\le \sup _{n \in {\mathbb {N}}} \sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma } \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{s} \sum _{l=0}^{p_j-1}1 \le C_s C_\gamma {\widehat{p}}. \end{aligned}$$

Since \(\rho \) is convex, we get that

$$\begin{aligned}&\rho (\eta M_{\gamma ,s,\alpha }(f)) \nonumber \\&\le \frac{1}{2} \rho ( 2\eta M_{\gamma ,s,\alpha }(f^{1})) + \frac{1}{2}\rho (2 \eta M_{\gamma ,s,\alpha }(f^{2})) \nonumber \\&\le \frac{1}{2}\int _{\varOmega _\infty ^{c}} \left( 2\eta M_{\gamma ,s,\alpha }(f^{1})(x)\right) ^{p(x)} \, dx + \eta \left\| M_{\gamma ,s,\alpha }(f^{1})\right\| _{L_\infty (\varOmega _\infty )} + 2 \eta C_sC_\gamma {\widehat{p}}, \end{aligned}$$
(4.5)

where \(8 \eta C_sC_\gamma {\widehat{p}}< \beta <1\) and \(\beta \) is given in (2.2). For a fixed kn, let us denote by \(\varLambda _{k,n}\) those triples (lji) for which \(0 \le j \le n, 0 \le i \le n\), \(l=0,\ldots ,p_j-1\) and

$$\begin{aligned} \frac{\beta }{\lambda (I_{k,n}^{l,j,i})^{1-\alpha }} \int _{I_{k,n}^{l,j,i}} f^{1}(t) \, dt \le 1. \end{aligned}$$

Then

$$\begin{aligned}&\frac{1}{2}\int _{\varOmega _\infty ^{c}} \left( 2\eta M_{\gamma ,s,\alpha }(f^{1})(x)\right) ^{p(x)} \, dx \\&\le \frac{1}{4}\int _{\varOmega _\infty ^{c}} \Bigg ( 4 \eta \sup _{n\in {{\mathbb {N}}}} \sum _{k=0}^{P_n-1} \chi _{I_{k,n}}(x) \sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma } \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{s} \\&\qquad \quad \sum _{l=0}^{p_j-1} \frac{\chi _{\varLambda _{k,n}}(l,j,i)}{\lambda (I_{k,n}^{l,j,i})^{1-\alpha }} \int _{I_{k,n}^{l,j,i}}f^{1}(t) \, dt\Bigg )^{p(x)} \, dx \\&\quad + \frac{1}{4}\int _{\varOmega _\infty ^{c}} \Bigg ( 4 \eta \sup _{n\in {{\mathbb {N}}}} \sum _{k=0}^{P_n-1} \chi _{I_{k,n}}(x) \sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma } \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{s} \\&\qquad \quad \sum _{l=0}^{p_j-1} \frac{\chi _{\varLambda _{k,n}^{c}}(l,j,i)}{\lambda (I_{k,n}^{l,j,i})^{1-\alpha }} \int _{I_{k,n}^{l,j,i}}f^{1}(t) \, dt\Bigg )^{p(x)} \, dx \\&=: (A_1)+(A_2). \end{aligned}$$

It is easy to see that

$$\begin{aligned} (A_1) \le \frac{1}{4}\int _{\varOmega _\infty ^{c}} \left( \frac{4 \eta C_sC_\gamma {\widehat{p}}}{\beta }\right) ^{p(x)} \, dx \le \frac{1}{4} \frac{4 \eta C_sC_\gamma {\widehat{p}}}{\beta } \le 1. \end{aligned}$$
(4.6)

We denote by \(I_{k,n,l,j,i,1}\) (resp. \(I_{k,n,l,j,i,2}\)) those points \(x \in I_{k,n}\) for which \(p(x) \le p_-(I_{k,n}^{l,j,i})\) (resp. \(p(x) > p_-(I_{k,n}^{l,j,i})\)). Then

$$\begin{aligned} (A_2)&\le \frac{1}{8} \sum _{\mu =1}^{2}\int _{\varOmega _\infty ^{c}} \left( \sup _{n\in {{\mathbb {N}}}} \sum _{k=0}^{P_n-1} \chi _{I_{k,n}}(x) \sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma } \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{s} \sum _{l=0}^{p_j-1} \right. \\&\qquad \left. \frac{8 \eta \chi _{\varLambda _{k,n}^{c}}(l,j,i)\chi _{I_{k,n,j,i,\mu }}(x)}{\lambda (I_{k,n}^{l,j,i})^{1-\alpha }} \int _{I_{k,n}^{l,j,i}} f^{1}(t) \, dt\right) ^{p(x)} \, dx \\&=: (A_{21})+(A_{22}). \end{aligned}$$

Let \(u(x):=p(x)/p_0>1\) for some \(1<p_0<q_-<p_-\), where we will choose \(p_0\) near to 1 later. By convexity and by the disjointness of the sets \(I_{k,n}\) for a fixed n, we conclude

$$\begin{aligned} (A_{21})&\le \frac{1}{8}\int _{\varOmega _\infty ^{c}} \Bigg ( \sup _{n\in {{\mathbb {N}}}} \sum _{k=0}^{P_n-1} \chi _{I_{k,n}}(x) \Bigg (\sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma } \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{s} \sum _{l=0}^{p_j-1} \\&\qquad \frac{1}{C_sC_\gamma {\widehat{p}}} \frac{8 \eta C_sC_\gamma {\widehat{p}}\chi _{\varLambda _{k,n}^{c}}(l,j,i)}{\lambda (I_{k,n}^{l,j,i})^{1-\alpha }} \\&\qquad \chi _{I_{k,n,l,j,i,1}}(x) \int _{I_{k,n}^{l,j,i}} f^{1}(t) \, dt\Bigg )^{u(x)}\Bigg )^{p_0} \, dx \\&\le \frac{1}{8} \int _{\varOmega _\infty ^{c}} \Bigg ( \sup _{n\in {{\mathbb {N}}}} \sum _{k=0}^{P_n-1} \chi _{I_{k,n}}(x) \sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma } \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{s} \sum _{l=0}^{p_j-1} \\&\qquad \frac{1}{C_sC_\gamma {\widehat{p}}} \Bigg (\frac{\beta \chi _{\varLambda _{k,n}^{c}}(l,j,i)}{\lambda (I_{k,n}^{l,j,i})^{1-\alpha }} \chi _{I_{k,n,l,j,i,1}}(x) \int _{I_{k,n}^{l,j,i}} f^{1}(t) \, dt\Bigg )^{u(x)}\Bigg )^{p_0} \, dx. \end{aligned}$$

Using that \(u(x) \le u_-(I_{k,n}^{l,j,i})\) on \(I_{k,n,l,j,i,1}\) and

$$\begin{aligned} \frac{\beta }{\lambda (I_{k,n}^{l,j,i})^{1-\alpha }} \int _{I_{k,n}^{l,j,i}} f^{1}(t) \, dt >1, \end{aligned}$$

we get that

$$\begin{aligned}&(A_{21})\\&\le \frac{1}{8(C_sC_\gamma {\widehat{p}})^{p_0}} \int _{\varOmega _\infty ^{c}} \left( \sup _{n\in {{\mathbb {N}}}} \sum _{k=0}^{P_n-1} \chi _{I_{k,n}}(x) \sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma } \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{s} \sum _{l=0}^{p_j-1} \right. \\&\qquad \left. \left( \frac{\beta \chi _{\varLambda _{k,n}^{c}}(l,j,i) \chi _{I_{k,n,l,j,i,1}}(x)}{\lambda (I_{k,n}^{l,j,i})^{1-\alpha }} \int _{I_{k,n}^{l,j,i}} f^{1}(t) \, dt\right) ^{u_-(I_{k,n}^{l,j,i})}\right) ^{p_0} \, dx . \end{aligned}$$

Equality (4.3) implies that \(1/p_-=1/q_-- \alpha \) and \(1/p_-(I_{k,n}^{l,j,i})=1/q_-(I_{k,n}^{l,j,i})- \alpha \), or, equivalently,

$$\begin{aligned} p_-(I_{k,n}^{l,j,i}) (1- \alpha q_-(I_{k,n}^{l,j,i})) = q_-(I_{k,n}^{l,j,i}). \end{aligned}$$

We use Lemma 5 with \(r=q_-(I_{k,n}^{l,j,i})\) to see that

$$\begin{aligned}&\left( \frac{1}{\lambda (I_{k,n}^{l,j,i})^{1-\alpha }} \int _{I_{k,n}^{l,j,i}} f^{1}(t) \, dt\right) ^{u_-(I_{k,n}^{l,j,i})} \nonumber \\&\qquad \le \left( \frac{1}{\lambda (I_{k,n}^{l,j,i})} \int _{I_{k,n}^{l,j,i}} f^{1}(t) \, dt\right) ^{q_-(I_{k,n}^{l,j,i})/p_0}. \end{aligned}$$
(4.7)

Hence

$$\begin{aligned}&(A_{21})\\&\le \frac{1}{8(C_sC_\gamma {\widehat{p}})^{p_0}} \int _{\varOmega _\infty ^{c}} \left( \sup _{n\in {{\mathbb {N}}}} \sum _{k=0}^{P_n-1} \chi _{I_{k,n}}(x) \sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma } \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{s} \sum _{l=0}^{p_j-1} \right. \\&\qquad \left. \left( \frac{\beta \chi _{\varLambda _{k,n}^{c}}(l,j,i) \chi _{I_{k,n,l,j,i,1}}(x)}{\lambda (I_{k,n}^{l,j,i})} \int _{I_{k,n}^{l,j,i}} f^{1}(t) \, dt\right) ^{q_-(I_{k,n}^{l,j,i})/p_0}\right) ^{p_0} \, dx . \end{aligned}$$

Here we have also used that \(\beta <1\) and \(u_-(I_{k,n}^{l,j,i})> q_-(I_{k,n}^{l,j,i})/p_0\). We know that \(\mathrm{supp \, }f^{1} \subset \varOmega _\infty ^{c}\) because \(\rho (f^{1}) \le 1\). Since \(f^{1}=f \chi _{\{f > 1\}}\), we have that \(\Vert f\Vert _{q(\cdot )/p_0} \le 1\) and we can apply Lemma 4 with \(p(\cdot )=q(\cdot )/p_0\). By Lemma 4 and Theorem 4, we conclude

$$\begin{aligned} (A_{21})&\le \frac{1}{8(C_sC_\gamma {\widehat{p}})^{p_0}} \int _{\varOmega _\infty ^{c}} \left( \sup _{n\in {{\mathbb {N}}}} \sum _{k=0}^{P_n-1} \chi _{I_{k,n}}(x) \sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma } \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{s} \right. \nonumber \\&\qquad \left. \sum _{l=0}^{p_j-1} \frac{1}{\lambda (I_{k,n}^{l,j,i})} \int _{I_{k,n}^{l,j,i}}|f^{1}(t)|^{q(t)/p_0} \, dt\right) ^{p_0} \, dx \nonumber \\&\le \frac{1}{8(C_sC_\gamma {\widehat{p}})^{p_0}} \left\| M_{\gamma ,s,\alpha }((f^{1})^{q(\cdot )/p_0}) \right\| _{p_0}^{p_0} \nonumber \\&\le C_1 \left\| |f|^{q(\cdot )/p_0} \right\| _{p_0}^{p_0} \le C_1. \end{aligned}$$
(4.8)

Choosing \(0<\gamma _0 < \gamma \) and \(0<r<s+\gamma _0\), we obtain

$$\begin{aligned}&(A_{22}) \nonumber \\&\le \frac{1}{8}\int _{\varOmega _\infty ^{c}} \Bigg ( \sup _{n\in {{\mathbb {N}}}} \sum _{k=0}^{P_n-1} \chi _{I_{k,n}}(x) \Bigg (\sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma - \gamma _0} \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{\gamma _0+s-r} \nonumber \\&\qquad \sum _{l=0}^{p_j-1} \frac{1}{C_{\gamma _0+s-r}C_{\gamma - \gamma _0} {\widehat{p}}} \left( \frac{P_i}{P_n} \right) ^{\gamma _0} \left( \frac{P_j}{P_i} \right) ^{r} \frac{8 \eta C_{\gamma - \gamma _0}C_{\gamma _0+s-r} {\widehat{p}} \chi _{\varLambda _{k,n}^{c}}(l,j,i)}{\lambda (I_{k,n}^{l,j,i})^{1- \alpha }} \nonumber \\&\qquad \chi _{I_{k,n,l,j,i,2}}(x)\int _{I_{k,n}^{l,j,i}} f^{1}(t) \, dt\Bigg )^{u(x)}\Bigg )^{p_0} \, dx \nonumber \\&\le \frac{1}{8}\int _{\varOmega _\infty ^{c}} \Bigg ( \sup _{n\in {{\mathbb {N}}}} \sum _{k=0}^{P_n-1} \chi _{I_{k,n}}(x) \sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma - \gamma _0} \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{\gamma _0+s-r} \nonumber \\&\qquad \sum _{l=0}^{p_j-1} \frac{1}{C_{\gamma _0+s-r}C_{\gamma - \gamma _0} {\widehat{p}}} \Bigg ( \left( \frac{P_j}{P_i} \right) ^{r} \frac{8 \eta C_{\gamma - \gamma _0}C_{\gamma _0+s-r} {\widehat{p}} \chi _{\varLambda _{k,n}^{c}}(l,j,i)}{\lambda (I_{k,n}^{l,j,i})^{1- \alpha }} \nonumber \\&\qquad \chi _{I_{k,n,l,j,i,2}}(x) \int _{I_{k,n}^{l,j,i}} f^{1}(t) \, dt\Bigg )^{u(x)}\Bigg )^{p_0} \, dx \nonumber \\&\le \frac{1}{8(C_{\gamma - \gamma _0}C_{\gamma _0+s-r} {\widehat{p}})^{p_0}}\int _{\varOmega _\infty ^{c}} \Bigg ( \sup _{n\in {{\mathbb {N}}}} \sum _{k=0}^{P_n-1} \chi _{I_{k,n}}(x) \sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma - \gamma _0} \nonumber \\&\qquad \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{\gamma _0+s-r} \sum _{l=0}^{p_j-1} \nonumber \\&\qquad \left( \beta ^{p_-} \left( \frac{P_j}{P_i} \right) ^{r} \frac{ \chi _{\varLambda _{k,n}^{c}}(l,j,i)\chi _{I_{k,n,l,j,i,2}}(x)}{\lambda (I_{k,n}^{l,j,i})^{1- \alpha }} \int _{I_{k,n}^{l,j,i}} f^{1}(t) \, dt\right) ^{u(x)}\Bigg )^{p_0} \, dx, \end{aligned}$$
(4.9)

where we can choose \(\eta \) such that \(8 \eta C_{\gamma - \gamma _0}C_{\gamma _0+s-r} {\widehat{p}} \le \beta ^{p_-}\). By (4.7), Hölder’s inequality and by the fact that \(\lambda (I_{k,n}^{l,j,i})=P_i^{-1}\),

$$\begin{aligned}&\left( \frac{1}{\lambda (I_{k,n}^{l,j,i})^{1-\alpha }} \int _{I_{k,n}^{l,j,i}} f^{1}(t) \, dt\right) ^{u(x)} \\&\le \left( \frac{1}{\lambda (I_{k,n}^{l,j,i})} \int _{I_{k,n}^{l,j,i}} f^{1}(t) \, dt\right) ^{\frac{q_-(I_{k,n}^{l,j,i})}{p_0} \frac{u(x)}{u_-(I_{k,n}^{l,j,i})}} \\&\le \left( \frac{1}{\lambda (I_{k,n}^{l,j,i})} \int _{I_{k,n}^{l,j,i}} |f^{1}(t)|^{q_-(I_{k,n}^{l,j,i})/p_0} \, dt\right) ^{u(x)/u_-(I_{k,n}^{l,j,i})} \\&\le P_i^{u(x)/u_-(I_{k,n}^{l,j,i})}\left( \int _{I_{k,n}^{l,j,i}} |f^{1}(t)|^{q_-(I_{k,n}^{l,j,i})/p_0} \, dt\right) ^{u(x)/u_-(I_{k,n}^{l,j,i})}. \end{aligned}$$

Consequently,

$$\begin{aligned}&(A_{22}) \nonumber \\&\le \frac{1}{8(C_{\gamma - \gamma _0}C_{\gamma _0+s-r} {\widehat{p}})^{p_0}}\int _{\varOmega _\infty ^{c}} \Bigg ( \sup _{n\in {{\mathbb {N}}}} \sum _{k=0}^{P_n-1} \chi _{I_{k,n}}(x) \sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma - \gamma _0} \nonumber \\&\qquad \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{\gamma _0+s-r} \sum _{l=0}^{p_j-1} \left( \frac{P_j}{P_i} \right) ^{ru(x)}\beta ^{p_-u(x)} P_i^{u(x)/u_-(I_{k,n}^{l,j,i})} \chi _{I_{k,n,l,j,i,2}}(x) \nonumber \\&\qquad \left( \int _{I_{k,n}^{l,j,i}}|f^{1}(t)|^{q_-(I_{k,n}^{l,j,i})/p_0} \, dt\right) ^{u(x)/u_-(I_{k,n}^{l,j,i})}\Bigg )^{p_0} \, dx. \end{aligned}$$
(4.10)

Recall that \(\mathrm{supp \, }f^{1} \subset \varOmega _\infty ^{c}\) and \(f^{1}>1\) or \(f^{1}=0\). Since \(u(x)>u_-(I_{k,n}^{l,j,i})\) on \(I_{k,n,l,j,i,2}\), \(q_-(I_{k,n}^{l,j,i})/p_0 \le q(t)\) for all \(t \in I_{k,n}^{l,j,i}\) and

$$\begin{aligned} \int _{I_{k,n}^{l,j,i}}|f^{1}(t)|^{q_-(I_{k,n}^{j,i})/p_0} \, dt \le \int _{\varOmega _\infty ^{c}}|f^{1}(t)|^{q(t)} \, dt \le 1, \end{aligned}$$
(4.11)

we can see that

$$\begin{aligned} (A_{22})&\le \frac{1}{8(C_{\gamma - \gamma _0}C_{\gamma _0+s-r} {\widehat{p}})^{p_0}}\int _{\varOmega _\infty ^{c}} \Bigg ( \sup _{n\in {{\mathbb {N}}}} \sum _{k=0}^{P_n-1} \chi _{I_{k,n}}(x) \sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma - \gamma _0} \nonumber \\&\qquad \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{\gamma _0+s-r} \sum _{l=0}^{p_j-1} \left( \frac{P_j}{P_i} \right) ^{ru(x)} P_i^{u(x)/u_-(I_{k,n}^{l,j,i})} \beta ^{p_-u(x)} \nonumber \\&\qquad \chi _{I_{k,n,l,j,i,2}}(x) \int _{I_{k,n}^{l,j,i}}|f^{1}(t)|^{q_-(I_{k,n}^{l,j,i})/p_0} \, dt \Bigg )^{p_0} \, dx \nonumber \\&\le \frac{1}{8(C_{\gamma - \gamma _0}C_{\gamma _0+s-r} {\widehat{p}})^{p_0}}\int _{\varOmega _\infty ^{c}} \Bigg ( \sup _{n\in {{\mathbb {N}}}} \sum _{k=0}^{P_n-1} \chi _{I_{k,n}}(x) \sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma - \gamma _0} \nonumber \\&\qquad \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{\gamma _0+s-r} \sum _{l=0}^{p_j-1} \left( \frac{P_j}{P_i} \right) ^{ru(x)} P_i^{u(x)/u_-(I_{k,n}^{l,j,i})-1} \beta ^{p_-u(x)} \nonumber \\&\qquad \frac{\chi _{I_{k,n,l,j,i,2}}(x)}{\lambda (I_{k,n}^{l,j,i})} \int _{I_{k,n}^{l,j,i}}|f^1(t)|^{q_-(I_{k,n}^{l,j,i})/p_0} \, dt\Bigg )^{p_0} \, dx. \end{aligned}$$
(4.12)

For fixed kn, let \(J_j\) denote the Vilenkin interval with length \(P_j^{-1}\) and \(I_{k,n} \subset J_j\). Then \(I_{k,n}^{l,j,i} \subset J_j\dot{+} lP^{-1}_{j+1} = J_j\). Inequality (2.2) implies that, for any \(x \in I_{k,n}\),

$$\begin{aligned} P_j^{-1/u(x)+1/u_-(I_{k,n}^{l,j,i})} \le \left( \frac{1}{\beta }\right) ^{p_0} \le \left( \frac{1}{\beta }\right) ^{p_-}, \end{aligned}$$

thus

$$\begin{aligned} \beta ^{p_-u(x)} \le P_j^{1-u(x)/u_-(I_{k,n}^{l,j,i})}. \end{aligned}$$
(4.13)

We can choose \(\gamma _0\), r and \(p_0\) such that

$$\begin{aligned} \frac{1}{u_-}-\frac{1}{u_+} <r. \end{aligned}$$
(4.14)

After an easy calculation, we can see that

$$\begin{aligned} ru(x) - \frac{u(x)}{u_-(I_{k,n}^{l,j,i})}+1&\ge u(x)\left( r - \frac{1}{u_-}\right) +1 >0. \end{aligned}$$

By this and (4.13),

$$\begin{aligned}&\left( \frac{P_j}{P_i} \right) ^{ru(x)} P_i^{u(x)/u_-(I_{k,n}^{l,j,i})-1} \beta ^{p_-u(x)}\\&\le \left( \frac{P_j}{P_i} \right) ^{ru(x)} P_i^{u(x)/u_-(I_{k,n}^{l,j,i})-1} P_j^{1-u(x)/u_-(I_{k,n}^{l,j,i})} \\&= \left( \frac{P_j}{P_i} \right) ^{ru(x) - \frac{u(x)}{u_-(I_{k,n}^{l,j,i})}+1} \le 1. \end{aligned}$$

We estimate (4.12) further by

$$\begin{aligned}&(A_{22}) \nonumber \\&\le \frac{1}{8(C_{\gamma - \gamma _0}C_{\gamma _0+s-r} {\widehat{p}})^{p_0}}\int _{\varOmega _\infty ^{c}} \left( \sup _{n\in {{\mathbb {N}}}} \sum _{k=0}^{P_n-1} \chi _{I_{k,n}}(x) \sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma - \gamma _0} \right. \nonumber \\&\quad \left. \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{\gamma _0+s-r} \sum _{l=0}^{p_j-1} \frac{\chi _{I_{k,n,l,j,i,2}}(x)}{\lambda (I_{k,n}^{l,j,i})} \int _{I_{k,n}^{l,j,i}}|f^1(t)|^{q_-(I_{k,n}^{l,j,i})/p_0} \, dt\right) ^{p_0} \, dx \nonumber \\&\le \frac{1}{8(C_{\gamma - \gamma _0}C_{\gamma _0+s-r} {\widehat{p}})^{p_0}}\int _{\varOmega _\infty ^{c}} \left( \sup _{n\in {{\mathbb {N}}}} \sum _{k=0}^{P_n-1} \chi _{I_{k,n}}(x) \sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma - \gamma _0} \right. \nonumber \\&\quad \left. \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{\gamma _0+s-r} \sum _{l=0}^{p_j-1} \frac{1}{\lambda (I_{k,n}^{l,j,i})} \int _{I_{k,n}^{l,j,i}}|f^1(t)|^{q(t)/p_0} \, dt\right) ^{p_0} \, dx \nonumber \\&\le \frac{1}{8(C_{\gamma - \gamma _0}C_{\gamma _0+s-r} {\widehat{p}})^{p_0}} \left\| M_{\gamma - \gamma _0,\gamma _0+s-r}(|f^1|^{q(\cdot )/p_0}) \right\| _{p_0}^{p_0} \nonumber \\&\le C_2 \left\| |f|^{q(\cdot )/p_0} \right\| _{p_0}^{p_0} \le C_2, \end{aligned}$$
(4.15)

whenever (4.14) holds. Since r can be arbitrarily near to \(s+ \gamma _0\), \(\gamma _0\) to \(\gamma \) and \(p_0\) to 1, (4.14) gives (4.4).

Now, we consider the second term of (4.5). We may suppose that \(\varOmega _\infty \) has positive measure and so \(p_+=\infty \). Let \(x \in \varOmega _\infty \) be fixed. Since

$$\begin{aligned} \eta M_{\gamma ,s,\alpha }(f^{1})(x)&\le \eta \sup _{n\in {{\mathbb {N}}}} \sum _{k=0}^{P_n-1} \chi _{I_{k,n}}(x) \sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma } \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{s} \\&\qquad \sum _{l=0}^{p_j-1} \frac{1}{\lambda (I_{k,n}^{l,j,i})^{1- \alpha }} \int _{I_{k,n}^{l,j,i}}f^{1}(t) \, dt, \end{aligned}$$

there exist \(n \in {{\mathbb {N}}}\) and \(0 \le k<2^{n}\) such that

$$\begin{aligned} \eta M_{\gamma ,s,\alpha }(f^{1})(x)&\le 2 \eta \chi _{I_{k,n}}(x) \sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma } \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{s} \\&\qquad \sum _{l=0}^{p_j-1} \frac{1}{\lambda (I_{k,n}^{l,j,i})^{1- \alpha }} \int _{I_{k,n}^{l,j,i}}f^{1}(t) \, dt. \end{aligned}$$

This means that \(x \in I_{k,n}\) and so \(p_+(I_{k,n})=p_+=\infty \). Let \(J_j\) denote the Vilenkin interval with length \(P_j^{-1}\) such that \(I_{k,n} \subset J_j\) \((j=0,\ldots ,n)\). Then \(p_+(J_j)=\infty \) and \(I_{k,n}^{l,j,i} \subset J_j\dot{+} lP^{-1}_{j+1} = J_j\). We may suppose that \(p_-(I_{k,n}^{l,j,i})<\infty \) for all \(j=0,\ldots ,n\), \(i=j,\ldots ,n\) and \(l=0,\ldots ,p_j-1\). Indeed, if \(p_-(I_{k,n}^{l,j,i})=\infty \), then \(I_{k,n}^{l,j,i} \subset \varOmega _\infty \). Since \(\mathrm{supp \, }f^{1} \subset \varOmega _\infty ^{c}\), this implies

$$\begin{aligned} \int _{I_{k,n}^{l,j,i}}f^{1}(t) \, dt=0. \end{aligned}$$

Let us choose \(v \in {\mathbb {R}}\) such that \(2^v \ge P_n\) and \(v \ge p_-(I_{k,n}^{l,j,i})\) for all \(j=0,\ldots ,n\), \(i=j,\ldots ,n\) and \(l=0,\ldots ,p_j-1\).

We can see as in (4.9) and (4.10) that

$$\begin{aligned}&(\eta M_{\gamma ,s,\alpha }(f^{1})(x))^{v}\\&\le \left( \sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma - \gamma _0} \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{\gamma _0+s-r} \sum _{l=0}^{p_j-1} \frac{1}{C_{\gamma _0+s-r}C_{\gamma - \gamma _0} {\widehat{p}}} \left( \frac{P_i}{P_n} \right) ^{\gamma _0} \right. \\&\qquad \left. \left( \frac{P_j}{P_i} \right) ^{r} \frac{2 \eta C_{\gamma - \gamma _0}C_{\gamma _0+s-r} {\widehat{p}} }{\lambda (I_{k,n}^{l,j,i})^{1- \alpha }} \int _{I_{k,n}^{l,j,i}} f^{1}(t) \, dt\right) ^v \\&\le \frac{1}{C_{\gamma _0+s-r}C_{\gamma - \gamma _0} {\widehat{p}}}\sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma - \gamma _0} \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{\gamma _0+s-r} \sum _{l=0}^{p_j-1} \\&\qquad \left( \left( \frac{P_j}{P_i} \right) ^{r} \frac{ \beta }{\lambda (I_{k,n}^{l,j,i})^{1- \alpha }} \int _{I_{k,n}^{l,j,i}} f^{1}(t) \, dt\right) ^v \end{aligned}$$

and

$$\begin{aligned}&(\eta M_{\gamma ,s,\alpha }(f^{1})(x))^{v} \\&\le \frac{1}{C_{\gamma _0+s-r}C_{\gamma - \gamma _0} {\widehat{p}}}\sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma - \gamma _0} \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{\gamma _0+s-r} \sum _{l=0}^{p_j-1} \\&\qquad \left( \frac{P_j}{P_i} \right) ^{rv} \beta ^v\left( \frac{ 1}{\lambda (I_{k,n}^{l,j,i})} \int _{I_{k,n}^{l,j,i}} |f^{1}(t)| \, dt\right) ^{q_-(I_{k,n}^{l,j,i}) v/p_-(I_{k,n}^{l,j,i})} \\&\le \frac{1}{C_{\gamma _0+s-r}C_{\gamma - \gamma _0} {\widehat{p}}}\sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma - \gamma _0} \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{\gamma _0+s-r} \sum _{l=0}^{p_j-1} \\&\qquad \left( \frac{P_j}{P_i} \right) ^{rv} \beta ^v \left( \frac{ 1}{\lambda (I_{k,n}^{l,j,i})} \int _{I_{k,n}^{l,j,i}} |f^{1}(t)|^{q_-(I_{k,n}^{l,j,i})} \, dt\right) ^{v/p_-(I_{k,n}^{l,j,i})}. \end{aligned}$$

Inequality (4.11) implies

$$\begin{aligned}&(\eta M_{\gamma ,s,\alpha }(f^{1})(x))^{v} \nonumber \\&\le \frac{1}{C_{\gamma _0+s-r}C_{\gamma - \gamma _0} {\widehat{p}}}\sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma - \gamma _0} \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{\gamma _0+s-r} \sum _{l=0}^{p_j-1} \nonumber \\&\qquad \left( \frac{P_j}{P_i} \right) ^{rv} \beta ^{v} P_i^{v/p_-(I_{k,n}^{j,i})} \left( \int _{I_{k,n}^{l,j,i}}|f^{1}(t)|^{q_-(I_{k,n}^{l,j,i})} \, dt\right) ^{v/p_-(I_{k,n}^{l,j,i})} \nonumber \\&\le \frac{1}{C_{\gamma _0+s-r}C_{\gamma - \gamma _0} {\widehat{p}}}\sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma - \gamma _0} \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{\gamma _0+s-r} \sum _{l=0}^{p_j-1} \nonumber \\&\qquad \left( \frac{P_j}{P_i} \right) ^{rv} P_i^{v/p_-(I_{k,n}^{j,i})-1} \beta ^{v} P_i \int _{I_{k,n}^{l,j,i}}|f^{1}(t)|^{q_-(I_{k,n}^{l,j,i})} \, dt. \end{aligned}$$
(4.16)

Since \(I_{k,n}^{l,j,i} \subset J_j\), we have \(p_-(J_j) \le p_-(I_{k,n}^{l,j,i}) \le v < p_+(J_j) = \infty \). Hence (2.2) implies

$$\begin{aligned} P_j^{-1/v+1/p_-(I_{k,n}^{l,j,i})} \le \frac{1}{\beta } \qquad \text{ and } \qquad \beta ^{v} \le P_j^{1-v/p_-(I_{k,n}^{l,j,i})}. \end{aligned}$$

Then

$$\begin{aligned} \left( \frac{P_j}{P_i} \right) ^{rv} P_i^{v/p_-(I_{k,n}^{l,j,i})-1} \beta ^{v}&\le \left( \frac{P_j}{P_i} \right) ^{rv} P_i^{v/p_-(I_{k,n}^{l,j,i})-1} P_j^{1-v/p_-(I_{k,n}^{l,j,i})} \\&= \left( \frac{P_j}{P_i} \right) ^{rv - \frac{v}{p_-(I_{k,n}^{l,j,i})}+1} \le 1 \end{aligned}$$

because

$$\begin{aligned} rv - \frac{v}{p_-(I_{k,n}^{j})}+1> rv - \frac{v}{p_-}+1 > 0. \end{aligned}$$

The last inequality holds since we can choose r such that

$$\begin{aligned} \frac{1}{p_-}<r. \end{aligned}$$

Recall that \(p_+=\infty \). This and inequality (4.16) imply

$$\begin{aligned}&(\eta M_{\gamma ,s,\alpha }(f^{1})(x))^{v} \\&\le \frac{1}{C_{\gamma _0+s-r}C_{\gamma - \gamma _0} {\widehat{p}}}\sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma - \gamma _0} \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{\gamma _0+s-r} \sum _{l=0}^{p_j-1} \\&\qquad P_i \int _{I_{k,n}^{l,j,i}}|f^{1}(t)|^{q_-(I_{k,n}^{l,j,i})} \, dt\\&\le \frac{1}{C_{\gamma _0+s-r}C_{\gamma - \gamma _0} {\widehat{p}}}\sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma - \gamma _0} \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{\gamma _0+s-r} \sum _{l=0}^{p_j-1} P_i \\&\le P_n. \end{aligned}$$

Thus

$$\begin{aligned} \eta M_{\gamma ,s,\alpha }(f^{1})(x)\le P_n^{1/v} <2 \qquad (x \in \varOmega _\infty ). \end{aligned}$$
(4.17)

Taking into account (4.5), (4.6), (4.8), (4.15) and (4.17), we obtain \(\rho (\eta M_{\gamma ,s,\alpha }(f)) \le C\), where \(C=4+C_1+C_2\). By convexity,

$$\begin{aligned} \rho \left( \frac{M_{\gamma ,s,\alpha }(f)}{C/\eta } \right) \le \frac{1}{C} \rho (\eta M_{\gamma ,s,\alpha }(f)) \le 1, \end{aligned}$$

which means that

$$\begin{aligned} \left\| M_{\gamma ,s,\alpha }(f)\right\| _{p(\cdot )} \le \frac{C}{\eta } = \frac{C}{\eta } \left\| f\right\| _{q(\cdot )}. \end{aligned}$$

This finishes the proof of Theorem 5. \(\square \)

Remark 2

Inequality (4.4) and Theorem 5 hold if \(p_->\max (1/(\gamma +s),1)\).

We point out the theorem for \(\alpha =0\):

Corollary 1

Let \(1/p(\cdot ) \in C^{\log }\), \(1<p_-\le p_+ \le \infty \) and \(0<\gamma ,s<\infty \). If (4.4) holds, then

$$\begin{aligned} \Vert M_{\gamma ,s,0}(f)\Vert _{p(\cdot )} \lesssim \Vert f\Vert _{{p(\cdot )}} \qquad (f\in L_{p(\cdot )}). \end{aligned}$$

Corollary 2

Let \(1/p(\cdot ) \in C^{\log }\) satisfy (4.4), \(1<p_-\le p_+ \le \infty \) and \(0<\gamma ,s<\infty \). Then, for all \(j=1,\ldots 4\) and \(f\in L_{p(\cdot )}\),

$$\begin{aligned} \Vert f\Vert _{p(\cdot )} \le \Vert M(f)\Vert _{p(\cdot )} \le \Vert M_{\gamma ,s,0}^{(j)}f\Vert _{p(\cdot )} \le \Vert M_{\gamma ,s,0}(f)\Vert _{p(\cdot )} \le C_{p(\cdot )} \Vert f\Vert _{{p(\cdot )}}. \end{aligned}$$

Proof

The inequalities follow from \(|f| \le M(f)\), (4.1) and Theorem 5. \(\square \)

In Theorem 7 we will show that condition (4.4) is important in Theorem 5, the result is not true without this condition.

5 The boundedness of fractional maximal operators on \(H_{p(\cdot )}\)

In this section, we apply the atomic characterization to prove the boundedness of \(M_{\gamma ,s,\alpha }\) from \(H_{q(\cdot )}\) to \(L_{p(\cdot )}\).

Theorem 6

Let \(1/q(\cdot ) \in C^{\log }\), \(0 \le \alpha <1\), \(0<q_-\le q_+ < 1/\alpha \) and \(0<\gamma ,s<\infty \). If (4.3) and (4.4) hold, then

$$\begin{aligned} \Vert M_{\gamma ,s,\alpha }(f)\Vert _{p(\cdot )} \lesssim \left\| f\right\| _{H_{q(\cdot )}} \qquad (f\in H_{q(\cdot )}). \end{aligned}$$

Proof

According to Theorem 2, f can be written as

$$\begin{aligned} f=\sum _{k \in {\mathbb {Z}}} \sum _{K\in {{\mathbb {N}}}} \sum _{\kappa } \mu _{k,K,\kappa } a^{k,K,\kappa } \quad \text{ with } \quad \mu _{k,K,\kappa }=3\cdot 2^k \Vert \chi _{I_{k,K,\kappa }}\Vert _{q(\cdot )} \end{aligned}$$

and

$$\begin{aligned} \left\| \left( \sum _{k \in {{\mathbb {Z}}}} \sum _{K\in {{\mathbb {N}}}} \sum _{\kappa } \left( \frac{\mu _{k,K,\kappa } \chi _{ I_{k,K,\kappa }}}{ \left\| \chi _{ I_{k,K,\kappa }} \right\| _{q(\cdot )}} \right) ^{t} \right) ^{1/t} \right\| _{q(\cdot )} \lesssim \left\| f \right\| _{H_{q(\cdot )}}, \end{aligned}$$

where \((a^{k,K,\kappa })_{k\in {\mathbb {Z}}, K,\kappa \in {\mathbb {N}}}\) is a sequence of \(q(\cdot )\)-atoms associated with the Vilenkin intervals \((I_{k,K,\kappa })_{k\in {\mathbb {Z}}, K,\kappa \in \mathbb N}\subset {\mathcal {F}}_K\), which are disjoint for fixed k, and \(0<t<{\underline{q}}\le {\underline{p}}\).

Then

$$\begin{aligned} \Vert M_{\gamma ,s,\alpha }(f)\Vert _{p(\cdot )}&\lesssim \left\| \sum _{k \in {{\mathbb {Z}}}} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } \mu _{k,K,\kappa } M_{\gamma ,s,\alpha }(a^{k,K,\kappa }) \chi _{I_{k,K,\kappa }} \right\| _{p(\cdot )} \\&\qquad +\left\| \sum _{k \in {{\mathbb {Z}}}} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } \mu _{k,K,\kappa } M_{\gamma ,s,\alpha }(a^{k,K,\kappa }) \chi _{I_{k,K,\kappa }^{c}} \right\| _{p(\cdot )}\\&=:Z_1+Z_2. \end{aligned}$$

We first estimate \(Z_1\). Since \(0<t <{\underline{q}} \le {\underline{p}}\le 1\), we have

$$\begin{aligned} Z_1\le \left\| \sum _{k \in {{\mathbb {Z}}}} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } \mu _{k,K,\kappa }^t M_{\gamma ,s,\alpha }(a^{k,K,\kappa })^t \chi _{I_{k,K,\kappa }} \right\| _{\frac{p(\cdot )}{t}}^{\frac{1}{t}}. \end{aligned}$$

By Lemma 1, we may choose \(g\in L_{(\frac{p(\cdot )}{t})'}\) with norm less than 1 such that

$$\begin{aligned}&\left\| \sum _{k \in {{\mathbb {Z}}}} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } \mu _{k,K,\kappa }^t M_{\gamma ,s,\alpha }(a^{k,K,\kappa })^t \chi _{I_{k,K,\kappa }}\right\| _{\frac{p(\cdot )}{t}} \\&\qquad = \int _0^1 \sum _{k \in {{\mathbb {Z}}}} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } \mu _{k,K,\kappa }^t M_{\gamma ,s,\alpha }(a^{k,K,\kappa })^t \chi _{I_{k,K,\kappa }} g \, d \lambda . \end{aligned}$$

Choose \(r>1\) such that \(rt>\max (p_+, 1/(1- \alpha ))\). By Hölder’s inequality we get that

$$\begin{aligned} Z_1^t&\le \int _0^1 \sum _{k \in {{\mathbb {Z}}}} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } \mu _{k,K,\kappa }^t M_{\gamma ,s,\alpha }(a^{k,K,\kappa })^t \chi _{I_{k,K,\kappa }} g \, d \lambda \\&\le \sum _{k \in {{\mathbb {Z}}}} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } \mu _{k,K,\kappa }^t \left\| M_{\gamma ,s,\alpha }(a^{k,K,\kappa })^t \right\| _{r} \left\| \chi _{I_{k,K,\kappa }} g \right\| _{r'}. \end{aligned}$$

Theorem 5 yields that \(M_{\gamma ,s,\alpha }\) is bounded from \(L_{v}\) to \(L_{rt}\), where v is defined by \(1/rt=1/v- \alpha \). The inequality \(rt>1/(1- \alpha )\) implies that \(1<v<1/\alpha \). Hence

$$\begin{aligned} \left\| M_{\gamma ,s,\alpha }(a^{k,K,\kappa })^t \right\| _{r}&= \left\| M_{\gamma ,s,\alpha }(a^{k,K,\kappa })\right\| _{rt}^t \lesssim \left\| a^{k,K,\kappa }\right\| _{v}^t \\&\le \left\| a^{k,K,\kappa }\right\| _{\infty }^t \lambda (I_{k,K,\kappa })^{t/v} \\&= \left\| \chi _{I_{k,K,\kappa }}\right\| _{q(\cdot )}^{-t} \lambda (I_{k,K,\kappa })^{1/r+ \alpha t}. \end{aligned}$$

Using the definition of \(\mu _{k,K,\kappa }\), we estimate \(Z_1\) further by

$$\begin{aligned} Z_1^t&\lesssim \sum _{k \in {{\mathbb {Z}}}} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } (3\cdot 2^k)^{t} \lambda (I_{k,K,\kappa })^{1/r+ \alpha t} \left\| \chi _{I_{k,K,\kappa }} g \right\| _{r'}\\&\le \sum _{k \in {{\mathbb {Z}}}} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } (3\cdot 2^k)^{t } \lambda (I_{k,K,\kappa }) \left( \frac{1}{\lambda (I_{k,K,\kappa })^{1- \alpha tr'}} \int _{I_{k,K,\kappa }}g^{r'} d \lambda \right) ^{\frac{1}{r'}} \\&\le \sum _{k \in {{\mathbb {Z}}}} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } (3\cdot 2^k)^{t} \int _0^1 \chi _{I_{k,K,\kappa }} \left( M_{\alpha tr'}(g^{r'}) \right) ^{\frac{1}{r'}}d \lambda \\ {}&\le \left\| \sum _{k \in {{\mathbb {Z}}}} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } (3\cdot 2^k)^{t} \chi _{I_{k,K,\kappa }} \right\| _{\frac{q(\cdot )}{t}} \left\| \left( M_{\alpha tr'}(g^{r'}) \right) ^{\frac{1}{r'}} \right\| _{{(\frac{q(\cdot )}{t})'}}. \end{aligned}$$

Note that \(rt>p_+ \ge q_+\) implies that \(\left( q(\cdot )/t\right) '>r'\). This and equality (4.3) mean that

$$\begin{aligned} \frac{r'}{(q(\cdot )/t)'} = \frac{r'}{(p(\cdot )/t)'}- \alpha tr' \end{aligned}$$

and so we can apply Theorem 5 to obtain that \(M_{\alpha tr'}\) is bounded from \(L_{\frac{(p(\cdot )/t)'}{r'}}\) to \(L_{\frac{(q(\cdot )/t)'}{r'}}\). More precisely,

$$\begin{aligned} \left\| \left( M_{\alpha tr'}(g^{r'}) \right) ^{\frac{1}{r'}} \right\| _{{(\frac{q(\cdot )}{t})'}}&= \left\| M_{\alpha tr'}(g^{r'}) \right\| _{{(\frac{q(\cdot )}{t})'/r'}}^{\frac{1}{r'}} \\&\lesssim \left\| g^{r'} \right\| _{{(\frac{p(\cdot )}{t})'/r'}}^{\frac{1}{r'}} = \left\| g \right\| _{{(\frac{p(\cdot )}{t})'}} \le 1. \end{aligned}$$

Consequently,

$$\begin{aligned} Z_1&\lesssim \left\| \sum _{k \in {{\mathbb {Z}}}} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } (3\cdot 2^k)^{t} \chi _{I_{k,K,\kappa }} \right\| _{\frac{q(\cdot )}{t}}^{\frac{1}{t}} \\&= \left\| \left( \sum _{k \in {{\mathbb {Z}}}} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } (3\cdot 2^k)^{t} \chi _{I_{k,K,\kappa }}\right) ^{\frac{1}{t}} \right\| _{q(\cdot )} \lesssim \Vert f\Vert _{H_{q(\cdot )}}. \end{aligned}$$

Now we estimate \(Z_2\). The operator \(M_{\gamma ,s,\alpha }\) can also be written in the following way:

$$\begin{aligned}&M_{\gamma ,s,\alpha }(a^{k,K,\kappa })(x) \\&= \sup _{n \in {\mathbb {N}}} \sup _{x \in I} \sum _{m=0}^{n} \sum _{j=0}^{m} \left( \frac{P_j}{P_n} \right) ^{\gamma } \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{s} \sum _{l=0}^{p_j-1}\frac{1}{\lambda (I^{l,j,i})^{1- \alpha }}\left| \int _{I^{l,j,i}} a^{k,K,\kappa } d \lambda \right| , \end{aligned}$$

where \(I \in {\mathcal {F}}_n\) is a Vilenkin interval. We suppose that \(x \notin I_{k,K,\kappa }\). Since \(\int _{I_{k,K,\kappa }}a^{k,K,\kappa }\, d\lambda =0\), we have

$$\begin{aligned} \int _{I^{l,j,i}} a^{k,K,\kappa } \, d \lambda = 0 \end{aligned}$$

if \(i \le K\). Thus we can suppose that \(i > K\), and so \(n \ge m > K\). If \(x \notin I_{k,K,\kappa }\), \(x \in I\) and \(j \ge K\), then \(I^{l,j,i} \cap I_{k,K,\kappa } = \emptyset \). Therefore we can suppose that \(j < K\). Similarly, if

$$\begin{aligned} x \in I_{k,K,\kappa } \dot{+} [l P_{j+1}^{-1}, (l+1) P_{j+1}^{-1}) \setminus (I_{k,K,\kappa } \dot{+} l P_{j+1}^{-1}), \end{aligned}$$

then \(I^{l,j,i} \cap I_{k,K,\kappa } = \emptyset \), so we may assume that \(x \in I_{k,K,\kappa }\dot{+}l P_{j+1}^{-1}=I_{k,K,\kappa }^{l,j,K}\). Therefore, for \(x \not \in I_{k,K,\kappa }\),

$$\begin{aligned} M_{\gamma ,s,\alpha }(a^{k,K,\kappa })(x)&\le \sup _{n> K} \chi _{I}(x) \sum _{m=K+1}^{n} \sum _{j=0}^{K-1} \left( \frac{P_j}{P_n} \right) ^{\gamma } \sum _{i=K+1}^{m} \left( \frac{P_j}{P_i} \right) ^{s} \\&\qquad \sum _{l=0}^{p_j-1} \frac{1}{\lambda (I^{l,j,i})^{1- \alpha }} \left| \int _{I^{l,j,i}} a^{k,K,\kappa } \, d \lambda \right| \chi _{I_{k,K,\kappa }^{l,j,K}}(x) \\&\le \Vert \chi _{I_{k,K,\kappa }}\Vert _{q(\cdot )}^{-1} \sup _{n > K} \chi _{I}(x) \sum _{m=K+1}^{n} \sum _{j=0}^{K-1} \left( \frac{P_j}{P_n} \right) ^{\gamma } \\&\qquad \sum _{i=K+1}^{m} \left( \frac{P_j}{P_i} \right) ^{s} \frac{1}{P_i^{\alpha }} \sum _{l=0}^{p_j-1} \chi _{I_{k,K,\kappa }^{l,j,K}}(x). \end{aligned}$$

Since

$$\begin{aligned} \sum _{i=K+1}^{m} \left( \frac{1}{P_i} \right) ^{s}&= \sum _{i=K+1}^{m} \left( \frac{1}{P_K p_K \cdots p_{i-1}} \right) ^{s} \\&\le \sum _{i=K+1}^{m} \left( \frac{1}{P_K 2^{i-K}} \right) ^{s} \le C_s \left( \frac{1}{P_K} \right) ^{s}, \end{aligned}$$

we have

$$\begin{aligned}&M_{\gamma ,s,\alpha }(a^{k,K,\kappa })(x) \\&\lesssim \Vert \chi _{I_{k,K,\kappa }}\Vert _{q(\cdot )}^{-1} \sup _{n> K} \chi _{I}(x) \sum _{m=K+1}^{n} \sum _{j=0}^{K-1} \left( \frac{P_j}{P_n} \right) ^{\gamma } \frac{P_j^s}{P_K^{s+ \alpha }} \sum _{l=0}^{p_j-1} \chi _{I_{k,K,\kappa }^{l,j,K}}(x) \\&\lesssim \Vert \chi _{I_{k,K,\kappa }}\Vert _{q(\cdot )}^{-1} \sup _{n> K} (n-K) \left( \frac{P_K}{P_n} \right) ^{\gamma } \sum _{j=0}^{K-1} \left( \frac{P_j}{P_K} \right) ^{\gamma +s} \frac{1}{P_K^{\alpha }}\sum _{l=0}^{p_j-1} \chi _{I_{k,K,\kappa }^{l,j,K}}(x)\\&\lesssim \Vert \chi _{I_{k,K,\kappa }}\Vert _{q(\cdot )}^{-1} \sup _{n > K} (n-K) 2^{(K-n)\gamma } \sum _{j=0}^{K-1} \left( \frac{P_j}{P_K} \right) ^{\gamma +s} \frac{1}{P_K^{\alpha }}\sum _{l=0}^{p_j-1} \chi _{I_{k,K,\kappa }^{l,j,K}}(x). \end{aligned}$$

Since the function \(x \mapsto x2^{-\gamma x}\) is bounded, we obtain that

$$\begin{aligned} M_{\gamma ,s,\alpha }(a^{k,K,\kappa })(x)&\lesssim \Vert \chi _{I_{k,K,\kappa }}\Vert _{q(\cdot )}^{-1} \sum _{j=0}^{K-1} \left( \frac{P_j}{P_K} \right) ^{\gamma +s} \frac{1}{P_K^{\alpha }}\sum _{l=0}^{p_j-1} \chi _{I_{k,K,\kappa }^{l,j,K}}(x). \end{aligned}$$

Consequently,

$$\begin{aligned} Z_2&\le \left\| \sum _{k \in {{\mathbb {Z}}}} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } \mu _{k,K,\kappa }^t M_{\gamma ,s,\alpha }(a^{k,K,\kappa })^t \chi _{I_{k,K,\kappa }^{c}} \right\| _{p(\cdot )/t}^{1/t} \\&\lesssim \left\| \sum _{k \in {{\mathbb {Z}}}} 2^{kt} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } \sum _{j=0}^{K-1} \left( \frac{P_j}{P_K} \right) ^{(\gamma +s)t} \frac{1}{P_K^{\alpha t}}\sum _{l=0}^{p_j-1} \chi _{I_{k,K,\kappa }^{l,j,K}} \right\| _{p(\cdot )/t}^{1/t}, \end{aligned}$$

where \(0<t<{\underline{q}}\le {\underline{p}}\). Let us choose \(\max (1,p_+)<r<\infty \). By Lemma 1, we can find a \(g\in L_{(\frac{p(\cdot )}{t})'}\) with \(\Vert g\Vert _{(\frac{p(\cdot )}{t})'} \le 1\) such that

$$\begin{aligned} Z_2^t&\lesssim \int _0^{1} \sum _{k \in {{\mathbb {Z}}}} 2^{kt} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } \sum _{j=0}^{K-1} \left( \frac{P_j}{P_K} \right) ^{(\gamma +s)t} \frac{1}{P_K^{\alpha t}} \sum _{l=0}^{p_j-1} \chi _{I_{k,K,\kappa }^{l,j,K}} g \, d \lambda \\&\le \sum _{k \in {{\mathbb {Z}}}} 2^{kt} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } \sum _{j=0}^{K-1} \left( \frac{P_j}{P_K} \right) ^{(\gamma +s)t} \frac{1}{P_K^{\alpha t}} \sum _{l=0}^{p_j-1} \Vert \chi _{I_{k,K,\kappa }^{l,j,K}}\Vert _{{\frac{r}{t}}} \Vert \chi _{I_{k,K,\kappa }^{l,j,K}} g\Vert _{{(\frac{r}{t})'}} \\&\lesssim \sum _{k \in {{\mathbb {Z}}}} 2^{kt} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } \sum _{j=0}^{K-1} \left( \frac{P_j}{P_K} \right) ^{(\gamma +s)t} \sum _{l=0}^{p_j-1} \\&\qquad \int _0^{1} \chi _{I_{k,K,\kappa }}(x) \left( \frac{1}{\lambda (I_{k,K,\kappa }^{l,j,K})^{1- \alpha t(r/t)'}}\int _{I_{k,K,\kappa }^{l,j,K}} \left| g\right| ^{(\frac{r}{t})'} \, d \lambda \right) ^{1/(\frac{r}{t})'}dx. \end{aligned}$$

Here we have used the fact that \(\lambda (I_{k,K,m})=\lambda (I_{k,K,m}^{l,j,K})=P_K^{-1}\). By Hölder’s inequality,

$$\begin{aligned}&Z_2^t\\&\lesssim \int _0^{1} \sum _{k \in {{\mathbb {Z}}}} 2^{kt} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } \chi _{I_{k,K,\kappa }}(x) \sum _{j=0}^{K-1} \sum _{l=0}^{p_j-1} \left( \frac{P_j}{P_K} \right) ^{(\gamma +s)t(1/(\frac{r}{t})+1/(\frac{r}{t})')} \\&\qquad \left( \frac{1}{\lambda (I_{k,K,\kappa }^{l,j,K})^{1- \alpha t(r/t)'}}\int _{I_{k,K,\kappa }^{l,j,K}} \left| g\right| ^{(\frac{r}{t})'} d \lambda \right) ^{1/(\frac{r}{t})'}dx\\&\lesssim \int _0^{1} \sum _{k \in {{\mathbb {Z}}}} 2^{kt} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } \chi _{I_{k,K,\kappa }}(x) \left( \sum _{j=0}^{K-1} \sum _{l=0}^{p_j-1} \left( \frac{P_j}{P_K} \right) ^{(\gamma +s)t} \right) ^{1/(\frac{r}{t})} \\&\qquad \left( \sum _{j=0}^{K-1} \sum _{l=0}^{p_j-1} \left( \frac{P_j}{P_K} \right) ^{(\gamma +s)t} \frac{1}{\lambda (I_{k,K,\kappa }^{l,j,K})^{1- \alpha t(r/t)'}}\int _{I_{k,K,\kappa }^{l,j,K}} \left| g\right| ^{(\frac{r}{t})'} d \lambda \right) ^{1/(\frac{r}{t})'} dx\\&\lesssim \int _0^{1} \sum _{k \in {{\mathbb {Z}}}} 2^{kt} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } \chi _{I_{k,K,\kappa }}(x) \\&\qquad \left( \sum _{j=0}^{K-1} \sum _{l=0}^{p_j-1} \left( \frac{P_j}{P_K} \right) ^{(\gamma +s)t} \frac{1}{\lambda (I_{k,K,\kappa }^{l,j,K})^{1- \alpha t(r/t)'}}\int _{I_{k,K,\kappa }^{l,j,K}} \left| g\right| ^{(\frac{r}{t})'} d \lambda \right) ^{1/(\frac{r}{t})'} dx. \end{aligned}$$

The definition of \(M_{\gamma t,st, \alpha t(r/t)'}^{(3)}\) and Lemma 1 imply

$$\begin{aligned}&Z_2^t\\&\lesssim \int _0^{1} \sum _{k \in {{\mathbb {Z}}}} 2^{kt} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } \chi _{I_{k,K,\kappa }} \left( M_{\gamma t,st,\alpha t(r/t)'}^{(3)} \left( |g|^{(\frac{r}{t})'} \right) \right) ^{1/(\frac{r}{t})'} d \lambda \\&\le \left\| \sum _{k \in {{\mathbb {Z}}}} 2^{kt} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } \chi _{I_{k,K,\kappa }} \right\| _{{q(\cdot )/t}} \left\| \left( M_{\gamma t,st,\alpha t(r/t)'}^{(3)} \left( |g|^{(\frac{r}{t})'} \right) \right) ^{1/(\frac{r}{t })'}\right\| _{{(q(\cdot )/t)'}}\\&= \left\| \sum _{k \in {{\mathbb {Z}}}} 2^{kt} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } \chi _{I_{k,K,\kappa }} \right\| _{{q(\cdot )/t}} \left\| M_{\gamma t,st,\alpha t(r/t)'}^{(3)} \left( |g|^{(\frac{r}{t})'}\right) \right\| _{{(q(\cdot )/t)'}/(r/t)'}^{1/(r/t)'}. \end{aligned}$$

Inequality (4.4) is equivalent to

$$\begin{aligned} \frac{q_+-t}{q_+}-\frac{q_--t}{q_-} < (\gamma +s)t. \end{aligned}$$

Since r can be arbitrarily large, we can choose it such that

$$\begin{aligned} \frac{1}{\left( (q(\cdot )/t)'/(r/t)'\right) _-}-\frac{1}{\left( (q(\cdot )/t)'/(r/t)'\right) _+}&= \frac{r/(r-t)}{q_+/(q_+-t)} - \frac{r/(r-t)}{q_-/(q_--t)} \\&< (\gamma +s)t. \end{aligned}$$

By inequality (4.4),

$$\begin{aligned} \frac{(r/t)'}{(q(\cdot )/t)'}= \frac{(r/t)'}{(p(\cdot )/t)'}-\alpha t(r/t)', \end{aligned}$$

which means that we can apply Theorem 5 to obtain

$$\begin{aligned} \left\| M_{\gamma t,st,\alpha t(r/t)'}^{(3)} \left( |g|^{(\frac{r}{t})'}\right) \right\| _{{(q(\cdot )/t)'}/(r/t)'}^{1/(r/t)'}&\lesssim \left\| |g|^{(\frac{r}{t})'}\right\| _{{(p(\cdot )/t)'}/(r/t)'}^{1/(r/t)'} \\&= \left\| g\right\| _{{(p(\cdot )/t)'}} \le 1. \end{aligned}$$

Hence

$$\begin{aligned} Z_2&\lesssim \left\| \left( \sum _{k \in {{\mathbb {Z}}}} \sum _{K \in {{\mathbb {N}}}} \sum _{\kappa } (3\cdot 2^k)^{t} \chi _{I_{k,K,\kappa }}\right) ^{\frac{1}{t}} \right\| _{q(\cdot )} \lesssim \Vert f\Vert _{H_{q(\cdot )}}. \end{aligned}$$

This completes the proof. \(\square \)

Remark 3

Inequality (4.4) obviously holds if \(1/(\gamma +s) \le p_-\le p_+<\infty \). If \(p_-<1/(\gamma +s)\), then (4.4) is equivalent to

$$\begin{aligned} p_+ < \frac{p_-}{1-(\gamma +s)p_-}. \end{aligned}$$

In the special case \(\alpha =0\), the theorem reads as follows:

Corollary 3

Let \(1/p(\cdot ) \in C^{\log }\), \(0<p_-\le p_+ < \infty \) and \(0<\gamma ,s<\infty \). If (4.4) hold, then

$$\begin{aligned} \Vert M_{\gamma ,s,\alpha }(f)\Vert _{p(\cdot )} \lesssim \left\| f\right\| _{H_{p(\cdot )}} \qquad (f\in H_{p(\cdot )}). \end{aligned}$$

Now we generalize Corollary 2 and get that \(\Vert \cdot \Vert _{H_{p(\cdot )}}\sim \Vert M_{\gamma ,s,0}(\cdot )\Vert _{p(\cdot )}\).

Corollary 4

Let \(1/p(\cdot ) \in C^{\log }\), \(0<\gamma ,s<\infty \) and \(0<p_-\le p_+<\infty \). If (4.4) holds and \(j=1,\ldots ,4\), then

$$\begin{aligned} \Vert f\Vert _{H_{p(\cdot )}} \le \Vert M_{\gamma ,s,0}^{(j)}f\Vert _{p(\cdot )} \le \Vert M_{\gamma ,s,0}(f)\Vert _{p(\cdot )} \le C_{p(\cdot )} \Vert f\Vert _{H_{p(\cdot )}} \qquad (f\in H_{p(\cdot )}). \end{aligned}$$

Theorems 5 and 6 do not hold if (4.4) is not satisfied. More exactly, we show

Theorem 7

Let \(1/p(\cdot ) \in C^{\log }\). If (4.3) hold and

$$\begin{aligned} \frac{1}{p_-(I_{0,n-1})}-\frac{1}{p_+(I_{0,n}^{1,0,n})} > \gamma +s \end{aligned}$$
(5.1)

for all \(n \in {{\mathbb {N}}}\), then \(M_{\gamma ,s,\alpha }\) is not bounded from \(H_{q(\cdot )}\) to \(L_{p(\cdot )}\).

Recall that \(I_{0,n}:= [0,P_n^{-1})\) and

$$\begin{aligned} I_{0,n}^{1,0,n}:= I_{0,n} \dot{+} [0,P_{n}^{-1}) \dot{+} P_{1}^{-1}= [0,P_{n}^{-1}) \dot{+} P_{1}^{-1}. \end{aligned}$$

Proof

Let

$$\begin{aligned} a_{n-1}(t)=P_{n-1}^{1/q_-(I_{0,n-1})} ((p_{n-1}-1)\chi _{I_{0,n}}-\chi _{I_{0,n-1}}) \end{aligned}$$

and \(x \notin I_{0,n-1}\). By Lemma 3, \(a_{n-1}\) is a \(q(\cdot )\)-atom for all \(n \ge 1\), and so \(\left\| a_{n-1}\right\| _{H_{q(\cdot )}} \le 1\). Choosing \(m=n=N\), \(i=n\) and \(l=1\), we can see that

$$\begin{aligned} M_{\gamma ,s,\alpha }a_{n-1}&= \sup _{N\in {{\mathbb {N}}}} \sum _{k=0}^{P_N-1} \chi _{I_{k,N}}\sum _{m=0}^{N} \sum _{j=0}^{m} \left( \frac{P_j}{P_N} \right) ^{\gamma } \sum _{i=j}^{m} \left( \frac{P_j}{P_i} \right) ^{s} \\&\qquad \sum _{l=0}^{p_j-1} \frac{1}{\lambda (I^{l,j,i}_{k,N})^{1- \alpha }}\left| \int _{I^{l,j,i}_{k,N}\cap I_{0,n-1}} a_{n-1} d \lambda \right| \\&\ge \chi _{J} \sum _{j=0}^{n} \left( \frac{P_j}{P_N} \right) ^{\gamma +s} \frac{1}{\lambda (J^{1,j,n})^{1- \alpha }}\left| \int _{J^{1,j,n}\cap I_{0,n-1}} a_{n-1} d \lambda \right| , \end{aligned}$$

where \(J=I_{0,n}^{1,0,n}\). The terms except \(j=0\) are all 0, so

$$\begin{aligned} M_{\gamma ,s,\alpha }a_{n-1}&\ge \chi _{I_{0,n}^{1,0,n}} P_n^{-\gamma -s} \frac{1}{\lambda (I_{0,n})^{1- \alpha }}\left| \int _{I_{0,n}\cap I_{0,n-1}} a_{n-1} d \lambda \right| \\&\ge \chi _{I_{0,n}^{1,0,n}}(x) P_n^{-\gamma -s- \alpha } P_{n-1}^{1/q_-(I_{0,n-1})}(p_{n-1}-1). \end{aligned}$$

Then

$$\begin{aligned} \int _{0}^{1} M_{\gamma ,s,\alpha }a_{n-1}(x)^{p(x)} \,dx&\ge \int _{I_{0,n}^{1,0,n}} P_n^{-(\gamma +s+ \alpha )p(x)} P_{n-1}^{p(x)/q_-(I_{0,n-1})} \, dx \\&\ge C \int _{I_{0,n}^{1,0,n}} P_n^{p_+(I_{0,n}^{1,0,n})(1/q_-(I_{0,n-1})-(\gamma +s+ \alpha ))} \, dx \\&\ge C \int _{I_{0,n}^{1,0,n}} P_n^{p_+(I_{0,n}^{1,0,n})(1/p_-(I_{0,n-1})-(\gamma +s))} \, dx \\&= C P_n^{p_+(I_{0,n}^{1,0,n})(1/p_-(I_{0,n-1})-(\gamma +s))} P_n^{-1}, \end{aligned}$$

which tends to infinity as \(n\rightarrow \infty \) if (5.1) holds. \(\square \)