1 Introduction

The large amount of existing data from observations of astrophysical processes around compact objects, such as gravitational waves [1] and black hole shadows [2, 3], together with gravity theories can provide a useful tool to understand the nature of the gravitational interaction [4,5,6]. On the other hand, testing and constraining the parameters of the gravity models is a step forward toward discovering the unified field theory.

According to Lovelock’s theorem, in less than five dimensions the cosmological constant can only appear within general relativity [7]. A new approach that avoids Lovelock’s theorem was recently proposed for obtaining the solution in Einstein-Gauss-Bonnet (EGB) gravity in four- and three-dimensional (\(D = 4, D = 3\)) spacetime [8]. The approach is based on using the rescaling of the Gauss–Bonnet term such that the limit \(D\rightarrow 4\) (\(D\rightarrow 3\) ) does not diverge.

Much work has been reported in the literature on EGB gravity in \(D=4\). In particular, the authors of Ref. [9] studied the gravitational collapse in 4D EGB gravity and showed the similarity of spherical dust collapse to one in Einstein’s gravity. The effects of the GB coupling constant on the super-radiance in black hole spacetime were studied in [10]. The stability of linearized equations of motion were analyzed in Ref. [11], exploring the perturbations of the black hole event horizon in GB gravity. Reference [12] is devoted to studying the classical spinning test particle motion around a non-rotating black hole in 4D EGB gravity. The authors of Ref. [13] analyzed the motion of test particles along the geodesic around a 4D EGB black hole. Charged particle and epicyclic motions around a 4D EGB black hole immersed in an external magnetic field were also considered in Ref. [14].

The rotating analogue of a static black hole solution in 4D EGB gravity was obtained in [15]. Analyzing the scalar and electromagnetic perturbation around a 4D EGB rotating black hole, the authors of Ref. [16] investigated the strong cosmic censorship conjecture. Other properties of the rotating spacetime in a 4D EGB black hole were explored in Refs. [17,18,19,20,21]. The energetics and shadow of a higher-dimensional (i.e 6D) EGB black hole were studied in [22]. Also, the authors of Refs. [23, 24] investigated overspinning of a 6D EGB black hole and circular orbits around higher-dimensional EGB rotating black holes.

The thermodynamics, phase transition and Joule–Thomson expansion of a 4D Gauss–Bonnet AdS black hole [25], particle acceleration [26], thermodynamic geometry of the 4D EGB AdS black hole [27], the emergent universe scenario in 4D EGB gravity [28], the extended thermodynamics and microstructures of a 4D charged EGB black hole [29], the shadow of a rotating 4D EGB black hole [30] and gravitational lensing of a 4D EGB black hole [31, 32] have been widely studied in the literature.

However, there has been a discussion of the validity of the model. In particular, Ref. [33] discussed the problem of the existence of EGB theory in 4D spacetime. The main conclusion of Ref. [34] was that 4D EGB is not well defined. Other authors [35, 36] have also questioned the validity of the field equations in the case of 4D EGB gravity. At the same time, in the limiting case when \(D\rightarrow 4\), the higher-dimensional scattering amplitudes of GB theory differ from general relativity and may be caused by an additional scalar-tensor field [37]. Here, we plan to study the spacetime properties around the 3D Gauss–Bonnet BTZ black hole using the analysis of test particles and photon motion. This study may be useful for developing new tests of the 3D Gauss–Bonnet theory and obtaining corresponding constraints on parameters of the theory. Another interesting aspect of studying black holes in dimensions \(D < 4\) is that they may refute what is true for black holes in dimension \(D=4\) [38]. We use the solution obtained in Ref. [39] describing the 3D BTZ black hole in EGB theory gravity.

Test particle motion and photon trajectories are both useful tools for exploring spacetime properties and its structure in various gravity theories [40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61]. In particular, the motion of charged and magnetized particles around a black hole in the presence of an external magnetic field was studied extensively in Refs. [59, 62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81].

Photon motion and its deflection in a gravitational field are two of the main features of the metric theories of gravity. One may read the review of the effects of gravitational lensing in Refs. [82,83,84,85]. A number of works have been devoted to exploring gravitational lensing in weak and strong field regimes [86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104]. One of the consequences of the exploration of photon motion is that it leads to the shadow of black holes. The discovery of the image of the black hole shadow by the EHT team [2, 3] has been investigated in theoretical studies of this effect by various authors [105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133]. In the present study, we are keen to explore the gravitational lensing in 3D EGB gravity. In particular, we examine the effects of the GB parameter and the cosmological constant on the gravitational lensing in 3D EGB gravity.

The paper is organized as follows: Section 2 is devoted to the review of the 3D BTZ black hole solution in EGB theory. We study the test particle motion around the 3D BTZ black hole in EGB theory in Sect. 3. We explore photon motion and gravitational lensing in Sect. 4, and we conclude our discussion in Sect. 5. Throughout this work, we use a system of units in which \(G=c=1\). Greek indices are taken to run from 0 to 2, and Latin ones from 1 to 2.

2 3D Gauss–Bonnet BTZ black hole metric

In D dimensions, the action for the GB theory with scalar field \(\phi \) is given by

$$\begin{aligned} S= & {} \frac{1}{16\pi G}\int \sqrt{-g} d^Dx\left[ R-2\Lambda +\alpha \left( \phi L_\mathrm{GB}\right. \right. \nonumber \\&+\left. \left. 4G^{\mu \nu }\partial _{\mu }\phi \partial _{\nu }\phi -4\left( \partial \phi \right) ^2\square \phi +2\big (\left( \triangledown \phi \right) ^2\big )^2 \right) \right] , \end{aligned}$$
(1)

where \(\alpha \) refers to the GB coupling constant, and \(L_\mathrm{GB}\) is the GB term given by

$$\begin{aligned} L_\mathrm{GB}=R_{\mu \nu \lambda \delta }R^{\mu \nu \lambda \delta } - 4R_{\mu \nu }R^{\mu \nu }+R^2, \end{aligned}$$
(2)

with R being the scalar curvature. Recently, [8] proposed a new approach in which it is possible to obtain the GB contribution in \(D=4\) by rescaling the coupling constant. It was also shown by [39] that it is possible to obtain a \(D=3\) case of the theory with \(\phi =\ln (\frac{r}{\ell })\), where \(\ell \) is the constant of integration. It is worth noting that [39, 134] for \(D=3\), the GB term \(L_\mathrm{GB}\) vanishes.

The form of the GB theory in \(D=3\) is given by

$$\begin{aligned} \mathrm{d}s^2=-F(r)\mathrm{d}t^2+\frac{\mathrm{d}r^2}{F(r)}+r^2\mathrm{d}\varphi ^2, \end{aligned}$$
(3)

with

$$\begin{aligned} F(r)=-\frac{r^2}{2\alpha }\left( 1\pm \sqrt{1+\frac{4\alpha }{r^2}f_{E}}\right) , \end{aligned}$$
(4)

where

$$\begin{aligned} f_{E}=\frac{r^2}{l^2}-m, \end{aligned}$$
(5)

where m and l respectively refer to the integration constants with \(\Lambda =-1/l^2\). Note that this 3D GB theory has two separate black hole solutions, i.e. \(\pm \sqrt{1+\frac{4\alpha }{r^2}f_{E}}\). It is worth noting that in the spacetime of a 3D BTZ black hole in GB gravity, one can calculate the effective AdS radius from the condition \(F(r)=0\), which is expressed as

$$\begin{aligned} r_\mathrm{AdS}=\sqrt{-\frac{m}{\Lambda }}, \end{aligned}$$
(6)

which is independent of the GB coupling parameter, \(\alpha \), while in higher-curvature theories this radius is a function of the coupling parameter [13,14,15, 20, 21, 30]. From the above equation, it is clearly seen that the integration constant m is a positive quantity, as \(\Lambda \) takes negative values only. Note that this AdS radius coincides with the one for the 3D BTZ black hole in Einstein gravity in the limit \(J\rightarrow 0\)  [135]. It can be also seen that the spacetime metric allows the cosmological constant to take only negative values to have the real AdS radius corresponding to the cosmological horizon which tends to infinity in the case in which \(\Lambda \rightarrow 0\).

Here we restrict ourselves to the ‘minus’ branch of the solution, as it leads to a well-defined BTZ solution only in the limiting case of small \(\alpha \), i.e. when the metric function F(r) is expanded in the form [39]

$$\begin{aligned} \lim _{\alpha \rightarrow 0} F(r) = \frac{r^2}{l^2}-m-\frac{\alpha }{r^2}\left( \frac{r^2}{l^2}-m\right) ^2+O(\alpha ^2), \end{aligned}$$
(7)

which coincides with the one for the standard BTZ solution for small \(\alpha \) in Einstein gravity. However, in the limiting case of large r, the metric function F(r) yields

$$\begin{aligned} F(r) = \frac{r^2}{2\alpha }\left( \sqrt{1+\frac{4\alpha }{l^2}}-1\right) -\frac{m}{\sqrt{1+\frac{4\alpha }{l^2}}}+O(1/r^2). \end{aligned}$$
(8)

From the above expression, the coupling parameter can take negative values, i.e. \(\alpha >-l^2/4\), at large distances (see for example [39] ). However, this would not be the case in close vicinity of the 3D BTZ black hole. Therefore, we further consider positive values of \(\alpha \) to explore the properties of the 3D BTZ black hole in GB gravity.

3 Test particle motion

Here we consider test particle motion in the gravitational field of the (2+1)-dimensional BTZ black hole in GB gravity.

Fig. 1
figure 1

Radial dependence of the effective potential for massive particles moving around the GB (2+1)-dimensional BTZ black hole. \(V_{eff}\) is plotted for different values of GB parameter \(\alpha \) for given \(\Lambda =0\) in the left panel, and for different values of \(\Lambda \) for given \(\alpha =0.1\) in the right panel

Full size image
Fig. 2
figure 2

The radial dependence of the specific angular momentum for test particles around the GB (2+1)-dimensional BTZ black hole. Left panel: for the different values of parameter \(\alpha \) in the case of a vanishing cosmological constant, i.e. \(\Lambda =0\). Right panel: for different values of the cosmological constant in the case of fixed \(\alpha =0.5\)

Full size image

To study the motion of test particles in the vicinity of the BTZ black hole, we explore the Hamilton–Jacobi equation [136]

$$\begin{aligned} H \equiv \frac{1}{2}g^{\mu \nu }\left( \frac{\partial S}{\partial x^{\mu }} \right) \left( \frac{\partial S}{\partial x^{\nu }} \right) , \end{aligned}$$
(9)

with the action S and the coordinate three-vector \(x^\mu \). The Hamiltonian is a constant that can be set to \(H=k/2\) with \(k=-m^{\prime 2}\) (where \(m^{\prime }\) is the mass of the test particle).

Then, from the symmetry of the system, one can write the action S for the motion of test particles around the black hole in separable form as

$$\begin{aligned} S= \frac{1}{2}k\lambda -Et+L\varphi +S_{r}(r). \end{aligned}$$
(10)

Here, E and L refer to the energy and angular momentum of the particle, respectively, and the parameter \(\lambda \) is an affine parameter. From Eq. (10), we rewrite the Hamilton–Jacobi equation in the following form

$$\begin{aligned}&k=&-F(r)^{-1}E^2 + F(r) \left( \frac{\partial S_{r}}{\partial r}\right) ^2 + \frac{L^2}{r^2}. \end{aligned}$$
(11)

In the (2+1)-dimensional system there appear three independent constants of motion, i.e. E, L and k, which are specified in [136]. In this case, the corresponding constant related to the latitudinal motion of particles is not available as that of the properties of the BTZ spacetime. From Eq. (11), we obtain the radial equation of motion for particles as

$$\begin{aligned} \frac{1}{2}\dot{r}^{2} + V_\mathrm{eff}(r;\mathcal {L},\alpha ,\Lambda )=\mathcal {E}^2, \end{aligned}$$
(12)

where the dot denotes a derivative with respect to the proper time \(\tau \), and the radial function \(V_\mathrm{eff}(r;\mathcal {L},\alpha ,\beta )\) refers to the effective potential of the system which is given by

$$\begin{aligned} V_\mathrm{eff}(r;\mathcal {L},\alpha ,\Lambda )= & {} \frac{r^2}{2\alpha }\left( \sqrt{1-\frac{4\alpha \left( \Lambda r^2 +m\right) }{r^2}}-1\right) \nonumber \\&\times \left( 1+\frac{\mathcal {L}^2}{r^2}\right) , \end{aligned}$$
(13)

with the conserved constants per unit mass \(m^{\prime }\) given by \(\mathcal {E}=E/m^{\prime }\) and \(\mathcal {L}=L/m^{\prime }\) and \(k/m^{\prime 2}=-1\).

In Fig. 1 we demonstrate the radial dependence of the effective potential (13) for different values of the GB parameter \(\alpha \) and cosmological constant \(\Lambda \). From Fig. 1, with increasing \(\alpha \), the curves begin to decrease. However, we can see that the negative cosmological constant, i.e. \(\Lambda <0\), has the opposite effect with respect to the GB parameter \(\alpha \), thereby suggesting that the effect of the cosmological constant can prevent test particles from escaping or falling into the black hole. Since the situation is altered for the cosmological constant, test particles under the effect of \(\alpha \) and \(\Lambda \) can have stable circular orbits around the (2+1)-dimensional BTZ black hole in GB gravity.

Following effective potential (13), we turn to the study of stable circular orbits of test particles around the BTZ black hole in GB gravity. For test particles to be on stable circular orbits, we shall focus on the required conditions for which

$$\begin{aligned} V_\mathrm{eff}(r;\mathcal {L},\alpha ,\Lambda )= & {} \mathcal {E}^2, \end{aligned}$$
(14)
$$\begin{aligned} \frac{\partial V_\mathrm{eff}(r;\mathcal {L},\alpha ,\Lambda )}{\partial r}= & {} 0. \end{aligned}$$
(15)

From the above equations, the corresponding energy and angular momentum for test particles on stable circular orbits can be obtained. Further, we show the dependence on the GB parameter and cosmological constant of the angular momentum required for the test particles to maintain a stable circular orbit around the black hole, see Fig. 2. It is clearly shown in Fig. 2 that the stable circular orbit of particles shifts toward the large radii as a result of an increase in the value of \(\alpha \). On the other hand, large values of \(\alpha \) give rise to an increase in the value of \(\mathcal {L}\), showing that for particles to remain on a stable circular orbit, a more angular momentum for a larger \(\alpha \) is needed. The situation, changes, however, as the cosmological parameter has the opposite effect, thereby reducing the value of the angular momentum for the particle to maintain a stable circular orbit around a (2+1)-dimensional BTZ black hole in GB gravity.

Table 1 The values of the ISCO radius \(r_\mathrm{isco}\) are tabulated in the case of massive particles moving around a 3D GB black hole for different values of GB parameter \(\alpha \) and cosmological constant \(\Lambda \)
Full size table
Fig. 3
figure 3

The dependence of the ISCO radius on the GB parameter \(\alpha \) for different values of cosmological constant \(\Lambda \)

Full size image
Fig. 4
figure 4

The radial profile of \(\partial ^2 V_\mathrm{eff}/\partial r^2\) for test particles making a circular motion around a (2+1)-dimensional BTZ black hole in GB gravity is plotted for different values of GB parameter \(\alpha \). Left/right panels refer to \(\Lambda =0/- 0.001\), respectively. Whereas, in the case of the (2+1)-dimensional BTZ black hole in Einstein gravity, the situation is totally altered from the one in GB gravity where \(V_\mathrm{eff}^{\prime \prime }\) does not depend upon r, ( \(V_\mathrm{eff}^{\prime \prime }=-8\Lambda \)), and hence there occur no stable circular orbits for massive particles (see for example [135])

Full size image

Further, we explore the innermost stable circular orbit (ISCO), which is given by the auxiliary condition on the effective potential

$$\begin{aligned} \frac{\partial ^2 V_\mathrm{eff}(r;\mathcal {L},\alpha ,\Lambda )}{\partial r^2}=0. \end{aligned}$$
(16)

In the case of massive particles, the radius of the ISCO \(r_i\) is obtained from the minimum value of the angular momentum \(\mathcal {L}\) determined by \({ V_\mathrm{eff}^{\prime }(r;\mathcal {L},\alpha ,\Lambda )}=0\). For the existence of ISCOs, the above condition must always be satisfied. It is difficult to solve and analyze Eq. (16) analytically, and hence we explore the ISCO radius numerically (see Table 1). As can be seen from Table 1, the ISCO radius increases with an increase in the GB parameter \(\alpha \), while it decreases with an increase in the cosmological parameter \(\Lambda \). This behavior of the ISCO radius is also shown very clearly in Fig. 3. From Eq. (6), it can be seen that the AdS radius corresponding to the cosmological horizon is always greater than the radius of stable circular orbits of the test particles for the chosen values of the cosmological constant. It turns out that the resultant gravitational force becomes stronger as we increase the GB parameter, thereby indicating that it acts as an attractive charge, whereas an increase in the value of the cosmological constant weakens the gravity. Thus, for a massive particle to be under the combined effect of the GB parameter and cosmological constant, it makes sense to move at stable circular orbits around the (2+1)-dimensional BTZ black hole in GB gravity. This leads to an interesting question: What happens to a massive particle in the case in which the effect of the cosmological parameter is switched off? Could it fall into the black hole under the effect of the GB parameter? To settle this question, we shall analyze Eq. (16), whether it is always negative or not. To this end, we plot the radial profile of \(V_\mathrm{eff}^{\prime \prime }(r)\) for various values of the GB parameter, see Fig. 4. We note that in the case of \(\Lambda =0\), \(V_\mathrm{eff}^{\prime \prime }(r)< 0\) always at all values of r, thereby indicating that no stable circular orbits occur around the black hole having the GB parameter only. The situation, however, is overturned once the effect arising from the cosmological constant is included, showing that \(V_\mathrm{eff}^{\prime \prime }(r)> 0\) always at all r; see (Fig. 4 right panel). Or, in other words, the particles can be on stable circular orbits around a (2+1)-dimensional BTZ black hole in GB gravity. It is interesting that this behavior for stable circular orbits exhibits a striking difference from its Einstein counterpart, for which there exists no occurrence of stable orbits [135]. However, in the GB gravity case, the cosmological constant plays a crucial role in obtaining information about stable circular orbits of test particles around the BTZ black hole and its properties as well.

4 Photon motion and gravitational lensing

In this section, we consider the motion of a photon in the GB (2+1)-dimensional BTZ black hole spacetime. From the usual relation for the three-momentum (note that we are dealing with 3D spacetime) \(p_\mu p^\mu =k\), we have that for photons, one has to set \(k=0\). From the Hamilton–Jacobi formalism, we obtain the action S in the form

$$\begin{aligned} S= -Et+L\varphi +S_{r}(r), \end{aligned}$$
(17)

where E and L are the usual conserved quantities for the energy and angular momentum of the photon, respectively, and \(S_{r}\) is a function of r only. Now it is straightforward to obtain the Hamilton–Jacobi equation in the following form

$$\begin{aligned} k= & {} -\frac{E^2}{F(r)}+F(r) \left( \frac{\partial S_{r}}{\partial r}\right) ^2 + \frac{L^2}{r^2}. \end{aligned}$$
(18)

Then, from the separability of the action given in Eq. (18), we obtain the radial component of equations of motion for photons in the following form

$$\begin{aligned} \dot{r}^2=E^2-\frac{L^2}{r^2}F(r). \end{aligned}$$
(19)

To find the radii of circular orbits for given values of E and L, we can then simultaneously solve \(\dot{r}=\ddot{r}=0\), i.e.

$$\begin{aligned} \tilde{V}_\mathrm{eff}(r,E,L)=0, \quad \frac{\partial \tilde{V}_\mathrm{eff}(r,E,L)}{\partial r}=0, \end{aligned}$$
(20)

where the function \(\tilde{V}_\mathrm{eff}(r,E,L)\) is defined as

$$\begin{aligned} \tilde{V}_\mathrm{eff}(r,E,L)= & {} E^2-\frac{L^2}{r^2}F(r). \end{aligned}$$
(21)

In (3+1)-dimensional spacetime, for a photon sphere, one needs to solve \(\tilde{V}_\mathrm{eff}'=0\), but interestingly, it is not sufficient to find the photon sphere around the (2+1)-dimensional BTZ black hole in GB gravity. Therefore, one may determine it using the additional condition \(\tilde{V}_\mathrm{eff}''=0\). Here, this condition gives \(r_\mathrm{ph}\) implicitly as

$$\begin{aligned} 3 (4 \alpha \Lambda -1)r^2 +8 \alpha m =0. \end{aligned}$$
(22)

In the limit of \(\alpha \Lambda \ll 1\), we can write the approximate expressions for the photon orbit \(r_{ph}\) as

$$\begin{aligned} r_\mathrm{ph}= & {} \frac{2 \sqrt{6}}{3} \left( 1+2 \alpha \Lambda \right) \sqrt{\alpha m } +O(\Lambda ^2). \end{aligned}$$
(23)

This clearly shows that the parameter \(\alpha \) increases the radius of the photon orbits. From the equation of motion for the mass-less particle, we can write \(u_\mu =\frac{\partial S}{\partial x^\mu }\) (and \(u^\mu =g^{\mu \nu }u_\nu \)), which allows us to write an explicit form of the components of the ‘three’-velocity as

$$\begin{aligned} \dot{t}= & {} \frac{2 \alpha }{r^2 \left( \sqrt{-4 \alpha \Lambda -\frac{4 \alpha m}{r^2}+1}-1\right) } E\ , \end{aligned}$$
(24)
$$\begin{aligned} \dot{\phi }= & {} \frac{L}{r^2}\ , \end{aligned}$$
(25)
$$\begin{aligned} \dot{r}^2= & {} E^2+\frac{L^2 \left( 1-\sqrt{1-4 \alpha \Lambda -\frac{4 \alpha m}{r^2}}\right) }{2 \alpha }\ , \end{aligned}$$
(26)

where differentiation is made over the affine parameter. Let us introduce the so-called distance of closest approach \(r_0\), defined as the minimum distance between the central gravitating object and the trajectory of photons. From Eq. (26) we can obtain the value for this distance as

$$\begin{aligned} r_0=\frac{L^2 \sqrt{m}}{\sqrt{\alpha E^4+\Lambda L^4+E^2 L^2}}. \end{aligned}$$
(27)

The dependence of the distance of closest approach on the impact parameter \(b=L/E\) is plotted in Fig. 5. It is apparent from the figure that the increase in the GB parameter decreases the distance of closest approach slightly, while the increase in the absolute value of the cosmological constant shows the opposite effect. It can be interpreted as follows: since the increase in the negative value of the cosmological constant increases the repulsive force between the central object and any particle with positive energy, then the increase in such repulsive force should in turn increase the closest distance between the trajectory of a particle and the central object for the given impact parameter b, which is shown in the right panel of Fig. 5.

Fig. 5
figure 5

The dependence of the distance of closest approach of a photon on the impact parameter b

Full size image
Fig. 6
figure 6

The dependence of the bending angle of photon on the inverse of the distance of closest approach \(u_0\)

Full size image

Let us now investigate the bending angle of a photon moving in the spacetime of the BTZ black hole in GB gravity. To do so, we can introduce a new variable, \(u=1/r\), that simplifies our calculation to find the bending angle. We can write the equation of motion in terms of the new variable as

$$\begin{aligned} \frac{\mathrm{d}\phi }{\mathrm{d}u}=\frac{L}{\sqrt{E^2+\frac{L^2 \left( 1-\sqrt{1-4 \alpha \Lambda -4 \alpha m u^2}\right) }{2 \alpha }}}. \end{aligned}$$
(28)

The resultant bending angle then takes the following form

$$\begin{aligned} \delta =2 \int _{0}^{u_0}\frac{L}{\sqrt{E^2+\frac{L^2 \left( 1-\sqrt{1-4 \alpha \Lambda -4 \alpha m u^2}\right) }{2 \alpha }}} \mathrm{d}u-\pi . \end{aligned}$$
(29)

One can explore the case when the GB parameter \(\alpha \) is small (i.e. \(\alpha \Lambda \ll 1\)) and use the linear approximation in it. In this approximation, the integral above can be written as

$$\begin{aligned} \delta= & {} 2 \int _{0}^{u_0}\frac{L}{\sqrt{\Lambda L^2+L^2 m u^2+E^2}} \nonumber \\&-\frac{\alpha L^3 \left( \Lambda +m u^2\right) ^2}{2 \left( \Lambda L^2+L^2 m u^2+E^2\right) ^{3/2}} \mathrm{d}u-\pi . \end{aligned}$$
(30)

After integration, the bending angle becomes

$$\begin{aligned} \delta= & {} \frac{\frac{\left( L^2 (4-\alpha \Lambda )+3 \alpha E^2\right) \log \left( \sqrt{m} \sqrt{L^2 \left( \Lambda +m u_0^2\right) +E^2}+L m u_0\right) }{\sqrt{m}}-\frac{\alpha L u_0 \left( E^2 L^2 \left( 2 \Lambda +m u_0^2\right) +\Lambda L^4 \left( \Lambda +m u_0^2\right) +3 E^4\right) }{\left( \Lambda L^2+E^2\right) \sqrt{L^2 \left( \Lambda +m u_0^2\right) +E^2}}}{2 L^2} \nonumber \\&-\frac{\left( L^2 (4-\alpha \Lambda )+3 \alpha E^2\right) \log \left( \sqrt{m} \sqrt{\Lambda L^2+E^2}\right) }{2 L^2 \sqrt{m}}-\pi . \end{aligned}$$
(31)

The dependence of such bending angle on the parameter \(u_0\) is plotted in Fig. 6 for the chosen energy \(E=1\) and angular momentum \(L=3\) of the photon and different values of the GB parameter \(\alpha \) with cosmological constant \(\Lambda \).

One can easily see from the left figure that in the absence of the GB parameter, the bending angle increases monotonically when one increases the inverse of the closest approach distance, \(u_0\). When this parameter is included, we see that the bending angle increases until its first peak and then starts to decrease and becomes zero for some value of \(u_0\). In this case, the maximum deflection angle can be easily found by setting equal to zero the derivative of the deflection angle over \(u_0\), which in turn reduces to the equation

$$\begin{aligned} \frac{2 E^2 L-L^3 \left( \Lambda +m u_0^2\right) \left( \alpha \left( \Lambda +m u_0^2\right) -2\right) }{\left( L^2 \left( \Lambda +m u_0^2\right) +E^2\right) ^{3/2}}=0. \end{aligned}$$
(32)

The maximum point then becomes

$$\begin{aligned} u_0^*=\left[ \frac{\sqrt{2 \alpha E^2+L^2}}{\alpha L m}+\frac{1}{\alpha m}-\frac{\Lambda }{m}\right] ^\frac{1}{2}. \end{aligned}$$
(33)

5 Conclusions

In the present study, we have seen that the GB parameter has an opposite effect on the radii of stable circular orbits with respect to the cosmological constant. The GB parameter allows particles not to be on stable circular orbits, while the cosmological constant restores the stable orbits for particles around the BTZ black hole in 3D EGB gravity. It is interesting that in the case of \(\Lambda =0\), there occur no stable circular orbits around the BTZ black hole in 3D EGB gravity. In other words, the particle under the effect of \(\alpha \) alone either can escape to infinity or fall into the black hole. The situation, however, is overturned once the effect arising from the cosmological constant is taken into consideration. It thus appears that the cosmological constant in (2+1)-dimensional BTZ black hole spacetime in GB gravity plays a crucial role for the existence of stable circular orbits for particles around the black hole. Also note that the ISCO radius increases with the increase in the GB parameter, while it decreases with an increase in the cosmological constant.

Investigation of photon motion around the BTZ black hole in 3D GB gravity shows that the radius of the photon orbit increases with the increase in the GB parameter, while the cosmological constant with its negative value decreases it. The study of the bending angle of the photon approaching the central object from infinity shows that in the presence of the cosmological constant, its value increases monotonically in the absence of the GB parameter, while in the presence of the latter it reaches its peak corresponding to the specific value of the parameter \(u_0^*\) and then decreases. It is also shown that the increase in the GB parameter reduces the bending angle, while the increase in the absolute value of the negative cosmological constant increases this angle.

In a recent work, Hennigar et al. [137] obtained the charged and rotating black hole solutions in the novel 3D GB theory of gravity which is a generalization of the BTZ solution. Their charged metric is obtained in the Maxwell and Born–Infeld theories. In a separate work, we will analyze the motion of charged particles and photons in these newly obtained charged and rotating spacetimes in 3D GB gravity to examine the effects of the charge and rotation on the motion of particles.