Numerical experiments on column-wise recurrence formula to compute fully normalized associated Legendre functions of ultra-high degree and order

Abstract

When using column-wise recurrence formulae to compute modified normalized Legendre functions up to approximate degree and order 3200 in the IEEE double precision environment, the overflow problem occurs. In this paper, we analyze the causes of the overflow problem in detail. We analyze column-wise recurrence formulae for computing fully normalized associated Legendre functions (fnALFs), their first-order derivatives, and the definite integrals of $\bar{P}_{nm} \left( {\cos \theta } \right)\sin \theta$. From our tests, we claim that if fully normalized column-wise recurrence formulae are applied, the computational accuracies of fnALFs and their first-order derivatives up to complete degree and order 64,800 can reach at least 10⁻¹¹. The computational accuracies of fully normalized column-wise recurrence formulae for computing definite integrals of $\bar{P}_{nm} \left( {\cos \theta } \right)\sin \theta$ are almost 10 times higher than those of the X-number method up to degree 6000. From the statistical results, the computational efficiency of fully normalized column-wise recurrence formulae for computing fnALFs and their first-order derivatives are almost 1.5 times faster than those of the X-number method, and approximately 15–22% faster than those of the dynamical switching X-number method. However, the computational efficiency of definite integrals is slower than that of the X-number method and dynamical switching X-number method.