Spectral/hp element methods: Recent developments, applications, and perspectives

The spectral/hp element method combines the geometric flexibility of the classical h-type finite element technique with the desirable numerical properties of spectral methods, employing high-degree piecewise polynomial basis functions on coarse finite element-type meshes. The spatial approximation is based upon orthogonal polynomials, such as Legendre or Chebychev polynomials, modified to accommodate a C0 - continuous expansion. Computationally and theoretically, by increasing the polynomial order p, high-precision solutions and fast convergence can be obtained and, in particular, under certain regularity assumptions an exponential reduction in approximation error between numerical and exact solutions can be achieved. This method has now been applied in many simulation studies of both fundamental and practical engineering flows. This paper briefly describes the formulation of the spectral/hp element method and provides an overview of its application to computational fluid dynamics. In particular, it focuses on the use of the spectral/hp element method in transitional flows and ocean engineering. Finally, some of the major challenges to be overcome in order to use the spectral/hp element method in more complex science and engineering applications are discussed.


Introduction
Over the past few decades, computational fluid dynamics (CFD) has become increasingly powerful and has therefore been seen as the natural starting point to investigate a variety of mathematical and physical problems in science and engineering. Traditionally, "low-order methods" with up to second-order spatial accuracy have been widely adopted as the default implementation for the simulation of fluid flows, often based around the Reynolds averaged Navier-Stokes (RANS) equations. This approach has achieved a great deal of success in many applications due to its well-established robustness and efficiency. However, as the demands on the accuracy of CFD outputs have increased, such as requiring the solution of the unsteady flow equations in complex geometries, low-order methods are less able to provide the necessary level of precision in capturing transient dynamics, as compared to higher accuracy schemes for a given computational cost. Therefore, there is currently a great interest in the development and application of high-order methods such as the spectral/hp element discretisation.
Finite element methods are widely used across a broad range of engineering and scientific disciplines. They can be categorised into three classes [1,2] : (1) the classical h-version finite element method, (2) the spectral element method, or p-version finite element method, (3) the hp-version finite element method, called the spectral/hp element method. Once the computational domain is partitioned into a non-overlapping element set, the spectral/hp element method employs a "spectral-like" approach in each element, representing the solution through a basis of polynomials. Therefore, the spectral/hp element method combines the advantages of the spectral element method, in terms of the properties of accuracy and rapid convergence, with those of the classical h-version finite element method, that allows complex geometries to be effectively captured. Compared with traditional low-order finite element schemes, the spectral/hp element method can provide an arbitrary order of spatial-approximation accuracy under the assumption of sufficient smoothness of the solution. It therefore combines the advantages of the low-order finite element method family, whilst also providing an additional attractive higher-precision, approximation for solving partial differential equations [3,4] . Recent studies also indicate that the compact nature of the spectral/hp element method is well-placed to take advantage of modern multi-core and many-core computing hardware [5,6] . All of the above properties have positioned the spectral/hp element method as an attractive numerical strategy for obtaining highprecision solutions with a relatively low computational cost.
The spectral/hp element method is gaining increasing traction in the field of CFD [3] , and it has achieved great success in both direct and large eddy simulation (LES) of complex flows [7][8][9][10][11] . It has also been successfully applied to a broad range of other applications in various research fields, for instance, cardiac electro-physiology [12] , solid mechanics [13,14] , porous media [15] and oceanographic modelling [16] .
One of the challenges of these methods is that they are more challenging to implement than loworder methods. There are now, however, now a number of open-source packages which encapsulate the complexities of the method. One such package that the authors have been developing is Nektar++, a cross-platform spectral/hp element framework (http:// www.nektar.info) which has made high-order finite element methods more accessible to the broader community and can be used to solve a range of the emerging challenges in high-fidelity scientific computations [4] . It enables the construction of high-order discretisations for solving a wide range of partial differential equations, supporting hybrid meshes, for example triangles and quadrilaterals or tetrahedra, prisms, pyramids and hexahedra. The current software package includes a number of pre-written scalable solvers, including for incompressible flow, compressible flow, shallow water equations, acoustic perturbation equations and others.
In this paper, we present a review of the state-ofthe-art of the spectral/hp element method, and its applications. In Section 1, we briefly describe the spectral/hp element method and discuss the numerical advantages and robustness of the method in solving partial differential equations. In Section 2, we high light some of the implementation challenges of the method in order to achieve computational efficiency and discuss potential solutions through our efforts with Nektar++. In Section 3, we then provide a survey of the applications in transitional flows and waves (wave propagation and wave-body interaction) in ocean engineering to which the spectral/hp element method has been applied. Finally, in Section 4, we conclude with further discussions and some perspectives on future directions.

Spectral/hp element method
In this section, we provide an overview of the mathematical foundations of the spectral/hp element method. A more detailed derivation of the mathematical theory can be found in Ref. [3] but is beyond the scope of this paper.

Overview of method
Spectral/hp element discretisations are the underpinning approximation for both continuous and discontinuous Galerkin formulations and can be constructed in 1-D, 2-D and 3-D. To obtain a general view of the method we can consider the example in Fig. 1. At the bottom of this figure we observe a triangulation of the British Isles. In general, the computational mesh can comprise of a mix of different shaped elements which could be triangles and quadrilaterals in two dimensions, and tetrahedra, pyramids, prisms and hexahedra in three dimensions. Within each element we develop a polynomial expansion, as represented in the lower-right of Fig. 1, where we observe all of the expansion modes to represent a fourth-order polynomial using a modal or hierarchical expansion basis. For Continuous Galerkin (CG) expansions, the design of these elements typically has the property of being decomposed into boundary and interior modes so that 0 -C continuity can be achieved by matching boundary expansion modes that have support on the edges of the element. The interior modes are necessarily defined to be zero on the boundaries.
There are a number of ways of developing the polynomial expansion within an element i K . In the Fig. 1 Illustration of the construction of a 2-D fourth-order 0 -C continuous modal triangular expansion basis using a generalised tensor-product procedure example shown in Fig. 1 we illustrate the popular construction using a tensor product of 1-D expansions which are defined in the regular regions K and K  which can then be mapped to the physical elemental regions using the mapping ( )   . Tensor-based expansions are very popular in quadrilateral and hexahedral regions but require some modification for triangular or tetrahedral regions using a Duffy [17] or collapsed-coordinate system [3] . It is possible to define polynomial expansions that are hierarchical/modal in construction, much like a small-scale spectral expansion, or alternatively one can use a nodal basis where the expansion is defined at a specific set of points (often related to Gaussian quadrature) and only one expansion mode has a unit value, whilst all other expansions bases are zero, at this point. This provides a Kronecker delta property to the expansion which can be useful when evaluating non-linear products. Again, for triangular elements the construction of a nodal basis is more involved and typically requires the use of a generalised Vandemonde matrix to relate them to a known expansion which is often of a modal construction [18] . Once a polynomial expansion has been defined in each element, the approach used to "bolt together" these individual expansions defines the approximation method. For example, if one enforces a 0 -C continuous expansion by ensuring the polynomial expansion is continuous across elemental boundaries we obtain the classical CG method. Alternatively, if one does not directly enforce the expansions to be 0 -C continuous but ensures appropriate flux quantities are continuous between elements, one can construct a so-called Discontinuous Galerkin (DG) scheme [3,18,19] .

Outline of mathematical formulation
In general, we consider the numerical solution of partial differential equations (PDEs) of the form = 0 Lu on a domain  , which may be geometrically complex, for some solution u . Practically,  takes the form of a d -dimensional finite element mesh consisting of elements  [20] . For a continuous discretisation, we additionally impose continuity along element interfaces. For illustrative purposes, we assume that it can be expressed as follows: where ( , ) a   is a bilinear form, ( ) l  is a linear form and 1 ( ) To solve this problem numerically, we consider solutions in a finite-dimensional subspace V   1 ( ) H  and cast our problem as augmented with appropriate boundary conditions. For a projection which enforces continuity across elements, we impose the additional constraint that 0 V C   . We assume the solution can be represented  [17] is used and operations, such as calculating derivatives, map the coordinate system to the noncollapsed coordinate system.
A local polynomial basis is constructed on each reference element with which to represent solutions. A one-dimensional order-P basis is a set of polynomials ( ) p   , 0 p P   , defined on the reference segment.
The particular choice of basis is usually made based on its mathematical or numerical properties and may be modal or nodal in nature. For 2-D and 3-D regions, a tensorial basis may be used, where the polynomial space is constructed as the tensor-product of 1-D bases on segments, quadrilaterals or hexahedral regions. In particular, a common choice is to use a modified hierarchical Legendre basis, given as a function of one variable by which supports the aforementioned boundary-interior decomposition and therefore improves numerical efficiency when solving the globally assemble dsystem. Equivalently, p where ˆn u are the coefficients from Eq. (1), obtained through projecting u onto the discrete space. Therefore, we restrict our solution space to 1 : is the space of order -p polynomials on i K . Elemental contributions to the solution may be assembled to form a global solution through an assembly operator. In a CG setting, the assembly operator sums contributions from neighbouring elements to enforce the 0 -C continuity requirement. In a DG formulation, such mappings transfer flux values from the element interfaces into global solution vector. For smooth problems one of the benefits of a spectral/hp formulation is often quoted as the exponential rate of convergence one observes when increasing the polynomial order of the approximation, providing the solution has sufficient regularity. In practice, the good approximation properties are also realised in terms of the better diffusion and dispersion properties one observes as you increase the polynomial order. This is exemplified in Fig. 2 where we have taken an initial condition of a Gaussian and advected it under a rotational velocity field for one revolution. We then observe the solution of a DG approximation of the linear advection problem using a = 1 P , = 3 P and = 8 P order approximation with 128, 32 and 8 elements, respectively. We see from this simple test that the shape is far less diffused in the = 8 P problem when compared to the = 2 P discretisation, even though the overall number of degrees of freedom is comparable.
Recently, a comparison between high-order flux reconstruction and the industrial-standard solver STAR-CCM+, was made, in which the accuracy and computational cost of flux reconstruction (FR) methods in PyFR and STAR-CCM+, for a range of test cases including scale-resolving simulations of turbulent flow were investigated [21] . The results from both PyFR and STAR-CCM+ show that third-order schemes can be more accurate than second-order schemes for a given cost. Moreover, advancing to higher-order schemes on GPUs with PyFR was found to offer even further accuracy vs cost benefits relative to industrystandard tools. These demonstrate the potential utility of high-order methods on GPUs for scale-resolving simulations of unsteady turbulent flows.

Using the spectral/hp element method in marginally resolved problems
In the previous sections, we outlined the underlying formulation of the spectral/hp element methodology and we now describe some of the state-of-theart developments necessary for simulating problems which are only marginally resolved. These tend to arise at higher Reynolds numbers, which generally promote greater levels of turbulence that cannot realistically be captured, even on the latest parallel computers. This leads to the build-up of energy in the larger resolved scales, due to the inability of the discretisation to fully capture the smaller scales of the flow. This is addressed with two techniques. The first involves performing the numerical integration of nonlinear terms consistently. Even with this, non-linear interactions may still lead to an artificial build-up of energy in the smaller resolved scales. A second technique is then to apply a spectral filtering, known as spectral vanishing viscosity, to dampen such energies.

Dealiasing
Dealiasing strategies have been observed to be effective in enhancing the numerical stability when solving problems using the spectral/hp element method [22][23][24][25][26] . The errors are typically caused by insufficient quadrature employed in the Galerkin discretisation of nonlinear terms [3] . When the simulation is not sufficiently well-resolved, this leads to energy in the shorter length scales to be transferred back into the longer length scales. When simulations are wellresolved, the numerical errors created by insufficient quadrature are negligible [23,24] . However, in marginally resolved or under-resolved simulations, aliasing errors may significantly pollute the solution [23] . As discussed in Ref. [26], there are three kinds of aliasing sources: (1) PDE aliasing, which relates to quasi-linear and non-linear terms [23] .
We start by giving an example from incompressible viscous flow past a NACA 0012 wing tip at a Reynolds number of 6 = 1.2 10 Re  , originally discussed in Ref. [26], to demonstrate the aliasing error in the near surface region. Figure 3 illustrates the dynamics of the flow by showing the iso-contours of helicity with a = 3 P approximation. In the boundary region, prismatic elements are employed while tetrahedral elements are used in the rest of the domain. Figure 4, shows the first two kinds of aliasing errors. In Fig. 4(a), 30% aliasing error with respect to the magnitude of the nonlinear terms is observed and in Fig. 4(b) we show a close up of regions with high geometrical aliasing error near the wing surface where 600% error is observed. The aliasing errors in this numerical calculation significantly pollute the solution and cause it to very rapidly become numerically unstable. Fig. 3 Helicity for an incompressible viscous flow over a NACA 0012 wing [26] Fig. 4 PDE aliasing errors at 30% of the magnitude of the nonlinear terms (a) and close up regions near the wing surface showing geometric aliasing errors (b) [26] We can categorise dealiasing techniques into two strategies: (1) Local dealiasing: PDE-dealiasing through consistent integration of the nonlinear terms only [23,26] .
(2) Global dealiasing: PDE-and geometricaldealiasing through consistent integration of all the terms of the discretisation [26] .
The phrase consistent integration refers to polynomial nonlinearities, where it is effectively possible to know a priori the number of quadrature points necessary for the integration to be exact [23,27] . Therefore, the non-linearities can be consistently integrated. For non-polynomial functions, as typically arise in the Euler equations, the concept of consistent integration is not well defined since it is impossible to fully control the quadrature error. We refer to the reader to Ref. [26] for a more complete discussion of these strategies as well as a discussion of interface aliasing.

Spectral vanishing viscosity
As previously discussed, it is well known that spectral/hp element discretisations generally lead to approximations that have low dissipation and low dispersion [3] per degree of freedom when compared to lower-order methods. In solving advection-diffusion equations and nonlinear partial differential equations such as advection-dominated flows, at marginal resolutions, oscillations appear that may render the computation unstable [3] . Artificial viscosity has been used in may discretisation methods to suppress wiggles associated with high wavenumbers, for example hyperviscosity has been broadly and effectively used in simulations using the Fourier method. A related concept is the so-called spectral vanishing viscosity (SVV), which was originally proposed based on a second-order diffusion (convolution) operator for spectral Fourier methods [28] . The SVV concept was originally motivated through the inviscid Burgers equation where an additional diffusion term was added, i.e.
where ( 0)   is a viscosity amplitude and D  is a viscosity kernel, which may be nonlinear, can be a function of x and is only activated for high wave numbers. With a small amount of controlled dissipation, spectral accuracy can be retained. This approach was extended to the Navier-Stokes equations by Karamanos and Karniadakis [29] and Kirby and Sherwin [30] and has also been investigated and applied to large eddy simulation by Pasquetti [31,32] .
To provide some intuition about the influence of SVV, we show in Fig. 5 an example from Ref. [30] of applying SVV to the parabolic equation   and periodic boundary conditions. The initial condition is given by ( , , = 0) = sin( )sin( ) u x y t x y   , in which a perturbation is introduced. A total of sixteen quadrilateral elements (four per direction) with 15th order polynomials per element direction were used. The following "classical" 1-D SVV kernel is employed, where P is the total number of modes employed and cut P is the cutoff polynomial order. SVV with the kernel function D can be regarded as a low-pass filter. We see that the SVV dissipation added to the high mode numbers with respect to the spectral element discretisation does indeed yield dissipation at the global high wavenumber scales of the solution in Fig. 5. In applications, Karamanos and Karniadakis applied SVV to high Reynolds number turbulent flows and the method was viewed as an alternative LES approach [29] . Kirby and Karniadakis computed the artificial viscosity by incorporating the local strain and the Panton function and the methodology was called SVV-LES [33] . The SVV approach has been reasonably widely used to simulate turbulent flow and vortical flows [7,[30][31][32][33][34][35][36] . SVV has been explicitly regarded as a turbulent model of implementing implicit large eddy simulation (iLES) under the assumption "The action of subgrid scales on the resolved scales is equivalent to strictly dissipative action" [37] , even though SVV is not explicitly designed as a subgrid-scale model.
Recently, in order to achieve a better performance of SVV, the linear advection equation augmented with SVV has been analysed [38] . Dispersion and diffusion characteristics of the spectral/hp CG formulation with SVV-based stabilization are verified to display similar non-smooth features in flow regions where convection is much stronger than dissipation or vice-versa, owing to a dependency of the standard SVV operator on a local Péclet number = /

Pe ah 
where a is the advection speed, h is the mesh spacing and  is the base SVV magnitude [38] . A modification was proposed [38] to the traditional SVV scaling, which enforces a globally constant Péclet number. In this approach, the artificial dissipation strength parameter , where  is a globally fixed parameter used to adjust the dissipation strength, P would be the polynomial order used in each element and c is a measure of the advection velocity magnitude on each local element. Furthermore, the following "power" kernel function is proposed [38] where p denotes mode index and SVV P is now not an activation threshold, but a similar effect in terms of computing the viscous effects on the highest modes. The mesh Péclet number is now held constant globally. In addition, the "power kernel" function has been devised for the advocated SVV operator to provide a consistent increase in resolution power (per degree of freedom) when the polynomial order is increased-a feature not naturally achieved through the widely used "exponential kernel" introduced in Ref. [39].
In turbulence simulations using DG methods, there were early discussions [40,41] , highlighting that the scheme successfully predicted low-order statistics with fewer degrees of freedom (DoFs) than traditional numerical methods. More extensive assessments [42,43] indicated that DG can predict high-order statistics with accuracy comparable to that of spectral methods for an equivalent number of DoFs. However, there is still little understanding of why and how one can use the spectral/hp element method for under-resolved turbulence simulations when either spectral vanishing viscosity (as sometime employed in CG methods) or upwind fluxes (as naturally arise in DG methods) are used to provide dissipation at under-resolved scales. Recently, exploratory studies, focusing on the DG formulation, were undertaken to address how numerical dissipation affects the under-resolved scales [44,90] . By exploring DG's propagation characteristics directly from the dispersion-diffusion curves of a linear advection problem, a simple criterion (named "the 1% rule") was proposed to estimate the effective resolution of the DG scheme for a given hp approximation space.
Numerical experiments on Burger turbulence helped to clarify why the DG formulation can be suitable for under-resolved simulations of turbulence: the numerical discretisation is capable of resolving scales up to 1%  with good accuracy while dissipation is provided at the end of the energy spectrum in the form of numerical diffusion. A further study for improving SVV by the "1% rule" was conducted with a new SVV kernel function [38] . The "1% rule" enables the SVV to mimic the property of the DG method with upwind numerical fluxes. A CG formulation based on the suggested SVV operator and kernel function was investigated. Being able to emulate the upwind properties of a DG scheme within a CG approximation using SVV appears to have relatively robust properties which are attractive for the marginally resolved simulations and this approach is referred to as a "DGKernel" SVV within the Nektar++ framework discussed in the next section.

Implementation of spectral/hp element method
One of the challenges of enabling the spectral/hp element method to become more pervasive for indu-strial and environmental problems is the implementational complexity of the methods. One of the overarching aims of the Nektar++ project is to provide an efficient framework upon which a broad range of physical processes can be modelled using these approaches. With this in mind, we outline some of efficiency challenges associated with implementing these methods for practical use and detail some suitable solutions in the context of Nektar++.

Nektar++ overview
The design of Nektar++ is intended to mirror the mathematical formulation of the spectral/hp element method, from one-dimensional basis functions up to multi-dimensional, multi-element discretisations of complex geometries. The package consists of a tiered collection of libraries which implement different aspects of the formulation, as illustrated in Fig. 6. On top of this, domain-specific application solvers can be readily developed which specify the physical processes to be modelled, while leveraging the implementation of the discretisation provided by the libraries in a relatively transparent manner.
At the lowest level the LibUtilities implements 1-D bases of orthogonal polynomials which are used to construct the expansion bases on individual elements. Their analytical derivatives and the distributions of points used for performing Gaussian quadrature are also captured. On top of this, the reference elements are defined for tessellating 1-D, 2-D and 3-D domains in the StdRegions library. Nektar++ supports a range of elemental regions to enable maximum flexibility in defining meshes which can effectively capture complex geometries, including segments in 1-D, quadrilaterals and triangles in 2-D, and hexahedra, prisms, pyramids and tetrahedra in 3-D. For multi-dimensional regions expansion bases may be defined using natively 2-D bases (e.g., Lagrange polynomials at Fekete points), or though a tensor-product construction of 1-D polynomials. For each of these regions, core finite element operators, including backward transform, inner product and derivative operators are defined.
Constituent elements of the computational mesh are represented in the LocalRegions library through a mapping from the reference region to the physical coordinates of the element. These mappings are captured in the SpatialDomains library and can capture high-order geometric curvature in the element. These geometric factors are used to extend the finite element operators defined for the standard reference regions to the physical elements. The MultiRegions library encapsulates the construction of multi-element global expansions through the construction of assembly operators which identify corresponding degrees of freedom in the local and global representations. Through this mapping, global finite element operators are constructed. Global linear systems can then be Fig. 6 Diagrammatic representation of the libraries in Nektar++ [4] solved using a number of direct and iterative linear algebra techniques. Iterative approaches, such as the preconditioned conjugate gradient method, solve a system through the repeated action of the operator, providing scope for a range of performance optimisations, discussed in Section 2.3.
While the central concept captured in Nektar++ are high-order spectral/hp element spatial discretisations, high-order time integration algorithms are also implemented to allow for highly accurate transient simulations. These include fully explicit, implicit and mixed implicit-explicit (IMEX) schemes.

Combined spectral/hp-Fourier discretisation
While the spectral/hp element method exhibits some of the numerical characteristics of a pure spectral method, the use of the latter is still preferred in the case of regular geometries such as rectangles and cuboidal domains. This situation occurs widely in many, particularly fundamental, hydrodynamics problems. In Nektar++, we leverage this by allowing for a mixed spectral/hp-Fourier discretisation. In this configuration, geometrically homogeneous coordinate directions are represented with a truncated Fourier expansion, while coordinate directions with geometric complexity use the spectral/hp element discretisation. This has the effect of decoupling the individual Fourier modes, allowing the spectral/hp operator to be applied independently within each component of each Fourier mode.
The Fourier decomposition also enables parallel decomposition of the modes to different processes [45] . Combined with the parallelisation of the spectral/hp element discretisation this leads to a hybrid parallelism. An open challenge here is identifying the optimal parallelisation of such mixed-discretisation domains to achieve optimal performance.
The GlobalMappings library further extends this discretisation strategy by supporting geometries in which a coordinate direction can be transformed to a homogeneous representation by an analytical function. This enables more complex geometries to be discretised using this efficient formulation.

From h to p efficiently
One of the challenges in implementing the spectral/hp method is achieving computational performance across a broad range of polynomial orders. The classic approach to implementing linear finite element operators is to assemble the global form of the operator and act directly on the global degrees of freedom. This is performant since each mesh vertex has a high valency and the assembly operation dramatically reduces the size of the global linear system used within the iterative solver. With increasing polynomial order, this reduction is less and the memory indirection associated with the global operator leads to reduced through-put and degraded performance.
A number of studies [46][47][48][49] have concluded that for higher polynomial orders, one should instead scatter the global degrees of freedom back onto their local elemental representation and perform the action of the operator in an element-wise fashion using a matrixvector operation. This enables a more compact representation of the operator in memory, greater cachelocality in applying the operator and a reduction in overall memory usage. At very high polynomial orders, the size of the local operator grows as 4 0( ) P . A more efficient approach in these cases is to take advantage of the tensor-product construction of the operators to decompose the elemental operator into a sequence of 1-D operators, known as sum-factorisation. Algebraically, this corresponds to a sequence of matrix-matrix operations which, besides the reduction in floating operation count, can be generally more efficiently implemented.
Recalling the construction of the local elemental operators as the reference region operator, transformed under a geometric mapping, one can exploit the decomposition of the elemental operators into these constituent parts. The Collections library in Nektar++ exploits this decomposition on groups of similar elements (same shape and expansion basis) by applying first the geometric factors associated with the operator, before acting with the reference region operator on all of the elements in the group simultaneously [50] . This effectively replaces a sequence of matrix-vector operations with a single matrix-matrix operation, reducing data movement and improving floating-point performance. This decomposition can be applied both to the local elemental matrix representation as well as the sum-factorisation approach.

High-order mesh generation
For complex 3-D geometries, the generation of high-order curved meshes along solid surfaces is a challenging topic [51][52][53][54][55] . Approaches for generating unstructured high-order meshes, extending coarse linear grids to incorporate high-order curvature, are a Fig. 7 Splitting a reference prismatic element and applying the mapping to obtain a high-order layer of prisms from the macro-element [51] Fig. 8 Surface mesh of a NACA 0012 wing geometry at polynomial order = 5 P . The coarse prismatic boundary layer is highlighted in blue (a) and the refined boundary layer which uses three layers and a progression ratio = 3 r (b) [51] Fig. 9(a) Turbulent channel flow Fig. 9(b) Turbulent pipe flow relatively new development [52,56,57] . The main challenge is robustness, since near-wall curvature must be introduced in such a way as to prevent the generation of self-intersecting elements [51] . Recently, several methods have been proposed to cope with high-order curvilinear boundary layer meshing, namely (1) the isoparametric approach [51] , (2) the variational approach [53] , and (3) the thermo-elastic analogy [54] . In the isoparametric approach, a reference element is mapped to a physical element with the supplied curvature information. The generation of a boundary layer mesh is achieved by splitting the reference element along the wall normal coordinate and then applying the mapping from a reference element to a physical element yielding the desired boundary layer mesh as illustrated in Fig. 7. In Fig. 8, a mesh for the NACA0012 aerofoil profile is shown which is generated by the isoparametric approach [51] . In the variational approach, a functional which defines a measure of energy over mesh and takes the mesh displacement and its derivatives as its arguments, is minimised using a nonlinear optimisation strategy [53] . It is demonstrated that the variational framework is efficient for both mesh quality optimisation and untangling of invalid meshes. In order to circumvent the drawback of elastic-model based methods which are used to tackle element self-intersection, a thermoelastic analogy, as an extension of the elastic formulation, is proposed to "heat" or "cool" elements. The thermo-elastic formulation leads to an additional degree of robustness, which shows the potential to significantly improve meshing [54] . NekMesh, which is a mesh generation and manipulation utility bundled with Nektar++, supports these strategies for highorder mesh generation.

Applications
In this section, we illustrate some typical applications of spectral/hp element methods in fluid mechanics, typically in hydrodynamics. Some examples will be given through use of the pre-written solvers in Nektar++.

The incompressible Navier-Stokes equations
We first introduce the incompressible Navier-Stokes equations on a bounded domain  which allows one to solve the governing equations for viscous Newtonian fluids governed by with appropriate boundary conditions. In Nektar++, a velocity correction scheme is employed, which uses a splitting/projection method where the velocity and the pressure are decoupled [58] . In the original approach, a stiffly stable time integration scheme was proposed. Briefly, high-order splitting schemes comprise of three steps involving explicit evaluation of the non-linear terms, followed by the implicit solution of the pressure Poisson system and finally solving a series of Helmholtz problems to enforce the viscous terms and velocity boundary conditions. Time integration is handled using a generic time-stepping framework, utilising one of a number of implicit-explicit (IMEX) time-integration schemes, the details of which can be found in Ref. [59].

Transitional flows
In the past few decades, the spectral/hp element method has been applied to laminar and transitional flows [58,[60][61][62][63][64] . As discussed earlier, the advantages of the spectral/hp element method are its low numerical dissipation and dispersion errors, which allow indivi-dual flow structures to be accurately captured over long time periods. Combined with the development of effective stabilisation techniques (e.g., SVV or filterbased stabilisation [65] ), the spectral/hp element method has in most recent years proved effective in modelling turbulent flow [33,[66][67][68][69][70] .
Transitional problems are those in which turbulence dominates the flow domain, or in which the transition to turbulence is a fundamental aspect of the simulation. Here we provide a survey of the range of studies undertaken using the spectral/hp element method in relation to transitional flows, focusing on (1) turbulence, (2) separated flows, (3) hydrodynamic stability, and (4) vortical flows and high Reynolds number wingtip vortices.

Turbulent flows
As noted in Ref. [4], highly resolved turbulent simulations were traditionally undertaken using spectral methods, imposing strong geometrical restrictions. Spectral/hp element methods alleviate this constraint although, as discussed in Section 2, when the domain of interest has a geometrically homogeneous direction, a combination of the spectral/hp element method and the traditional spectral method is still particularly advantageous [45,71] .
A benchmark simulation of turbulent flow over a periodic hill has been performed using Nektar++. It is challenging to resolve accurately due to the detachment of the fluid from the smooth surface and the generation of a recirculation region downstream of the hill [4] . A 2-D mesh of 3 626 quadrilateral elements with = 6 P in the streamwise direction was constructed, and a Fourier pseudo-spectral method with 160 collocation points in the spanwise direction was employed to perform the simulation at a Reynolds number 2 800. Excellent agreement with the benchmark statistics was obtained. With the quasi-3D solver, turbulent channel and pipe flows have also been accurately simulated and validated [45] , these are illustrated in Fig. 9 and are used as benchmark cases in Nektar++.
Recently, a global mapping technique has been developed which allows the simulation of fluid flows with geometrically periodic variation along homogeneous directions [8,11] . A typical example is to simulate turbulent flows induced by a localised surface deformation in the boundary layer [11] . The computational domain is illustrated in Fig. 10(a) and the turbulent flow which forms downstream of the surface deformation is shown in Fig. 10(b). The freestream unit Reynolds number is 1.210 6 and the reference free-stream velocity is 18m/s. The laminar-turbulent transition is initialised by the nonlinear process of upstream Tollmien-Schlichting (TS) disturbance breakdown to turbulence downstream of the surface indentation. The disturbance is generated by a disturbance strip as illustrated in Fig. 10(a), which has a vibration frequency of 172 Hz and is used to mimic the TS disturbance by the receptivity mechanism. The numerical calculations of this case are validated with an experimental study for which excellent agreement is obtained. Fig. 10 An illustration of the computational domain for the turbulent simulation in a boundary layer (a) and laminar-turbulent transition triggered by TS instability in a boundary layer. The iso-surfaces indicate the different pressure levels [11] 3.2.2 Separated flows Flow separation can be one of the most important topics in fluid mechanics, due to its relevance to aerodynamic performance in many engineering applications. There are typically two kinds of separations: geometrical, where the flow separates from a sharp obstacle in the flow, and pressure-induced, depending on the pressure gradient over a smooth surface. For both kinds of separation, the spectral/hp element method has been used to accurately capture detachment and reattachment points, e.g., laminar flow in a channel expansion [72] , turbulent separation in a 3-D diffuser [25] , the flow around a wall-mounted square cylinder [73] , flows over periodic hills [74] , small separation bubbles induced by localised imperfections [11,75] . Practically, the studies of separated flows are always accompanied by turbulent flows due to the occurrence of sensitive instabilities induced by the geometrical discontinuity.
Recently, with the global mapping technique, direct numerical simulations of the flow around wings with spanwise waviness were investigated to explore its effect on the wing performance [9,76] . The geometries are based on a NACA0012 profile with a small modification to obtain zero-thickness trailing edge to overcome meshing challenges. The wavy geometries were obtained by applying the following coordinate transformation to the straight infinite wing Fig. 11 The geometry of a wavy wing with / = 0.1 h c and / = 0.5 d  [9] [9] where h is the waviness peak-to-peak amplitude,  is its wavelength, x is the physical coordinate in the cord direction, and x and z are the chordwise and spanwise coordinates in the computational domain in Fig. 11. At a very low Reynolds number 1 000, for moderate angles of attack, the waviness leads to decreases in both drag and lift forces, which lead to a decrease in the lift-to-drag ratio and a suppression of the fluctuating lift coefficient [9] . The physical mechanism behind these were explained. In Fig. 12(a), the recirculation regions are shown and the visualisation was made considering the regions where streamwise velocity is negative. In Fig. 12(b), the coloured contours represent spanwise velocity at the plane = x 0.33  , which is close to the location of the leading edge in the peak of the waviness at = 0.05 x  , depicting how the flow moves away from the waviness peak in the lower portion of the wing and it moves towards it in the upper part. A further study was undertaken at Reynolds numbers 10 000 and 50 000 for different attack angles through highly resolved direct numerical simulations [76] , which provides a better understanding of wing performance with the use of spanwise waviness.

Hydrodynamic stability
In addition to solving the fully non-linear incompressible Navier-Stokes equations in Nektar++, the solution of the linearised incompressible Navier-Stokes equations is also supported to enable global flow stability analyses to be performed with respect to a steady or a time-periodic base flow. This process identifies whether such flows are susceptible to a fundamental change of state when an infinitesimal disturbance is introduction. The linearised incompressible Navier-Stokes equations take the following form where  u is the perturbation and U denotes the base flow or a time-dependent periodic flow which are sampled at regular intervals and interpolated. For stability analysis, suitable boundary conditions should be imposed. The linear evolution of a perturbation under Eq. (13) can be expressed as with an initial condition (0)  u . If the base flow U is steady, the perturbation ( ) t  u be expressed by eigenmode solutions of ( ) A t as follows The dominant eigenvalues and eigenmodes of the operator ( ) A t are defined by solutions to the following equation The leading eigenvalue j  is used to detect the global stability of the flow. Using a similar approach, the operator ( ) A t  for the adjoint form of the lineari-sed Navier-Stokes evolution operator can be used to examine the receptivity of the flow and, in combination with the direct mode, identify the sensitivity to base flow modification and local feedback. The direct and adjoint methods can be combined to explore convective instabilities over different time horizons  in a flow by computing the leading eigenmodes of ( )( ) known as transient growth analysis. The eigenvalues and eigenmodes of ( ) can be solved directly if the dimensions of the matrices are small. Practically, timestepper-based methodologies, based on Arnoldi iteration methods [77] , can be more efficient for complex problems [78][79][80][81] than direct calculation of the eigenvalue spectrum from the linearised Navier-Stokes operator.
To illustrate the direct stability analysis with a classic example, we calculated the dominant mode of the two-dimensional flow past a circular cylinder at The leading eigenmode is characterised by the asymmetry in the streamwise component and symmetry in the cross-stream component. We also observe the spatial distribution of the modes, with the leading direct modes extending far downstream of the cylinder. Recently, instability methods using the spectral/hp element method have been employed for investigating the stability of vortical flows and controlling wakes of flows past bluff bodies [82][83][84] . With the linearised Navier-Stokes equations, the interaction between instability waves and surface distortion in a 2-D and 3-D boundary layer have been precisely investigated [10,11,75] .

Vortical flows and wingtip vortex
Simulating and understanding vortical flows are vital in hydrodynamics and aerodynamics. In many applications, manipulation of vortices in the vicinity of flow boundaries is crucial for improving performance in engineering practice [7,[85][86][87][88][89] . Developing a better understanding of the near wake of the vortex, lying within one chord length of the trailing edge, is essential in understanding the complex flow structure interactions [7] , which is also crucial in understanding cavitation in vortical structures generated by hydro propellers [86] . Computationally, in high Reynolds number flows, accurate numerical simulations of these kinds of vortical structures are challenging for the traditional numerical methods due to numerical dissipation. A recent study in which wingtip vortices were simulated demonstrated that the adjustable and controllable low-dissipation properties of the spectral/ hp element method were beneficial for modelling and simulating vortical flows [7,44,90] .
To illustrate the ability of the spectral/hp element method to accurately capture vortical flows at a Reynolds number 1.210 6 , a rectangular wing with a NACA0012 profile, is investigated with a rounded wing cap (consequently, a longer semispan where the wing is thickest) and a blunt trailing edge [7] . The results showed better correlation with experimental results than previous numerical results, both in terms of the static pressure distribution, prediction of the jetting velocity, vortex spanwise location, and the ability to resolve the secondary vortex interaction with the main wingtip vortex. The iLES method based on SVV has been shown to be a compelling alternative for computing complex vortex-dominated flows, such as the wingtip vortex, motivating its use for complex industrially relevant cases where high-fidelity computational fluid dynamics can become an enabling technology. Built on the confidence of simulating the NACA0012 wingtip vortex, recently, an initial simulation with a Reynolds number 1.21710 6 is currently being conducted to simulate the tip vortex of an elliptic hydrofoil where the incident flow attack angle is o 7 . The tip vortex structure is illustrated by the iso-surfaces of the total pressure in Fig. 14. The contour figures of the vorticity components are shown at the section / = 1.06 x c in Fig. 15 and the origin is located at the tip of the hydrofoil. In Fig. 15(a), the vortex core is shown in axial vorticity This calculation will be used to provide vortex core pressure for studying cavitation in a vortex structure.
The numerical results will be going to be compared with experimental data.

Waves in ocean engineering
This section is devoted to outlining recent progress in spectral/hp element modelling of wave propagation and wave-body interaction. For water waves propagation, the fully nonlinear potential flow (FNPF) equations is the fundamental governing equation and can be derived from the Navier-Stokes equations by assuming inviscid and irrotational flow. One of the main challenges in the last decade have focused on use of spectral/hp element methods for coastal engineering [91] , and the development of robust and efficient solvers with support for unstructured meshes to capture realistic shore lines, geometric features, and adapt meshes to relevant features of the solution [92] .

Fully nonlinear potential flow
Finite elements are widely used for solving the FNPF equations [93][94][95][96] , but the use of spectral/hp elements remains scarse. The first attempt to solve the FNPF equations using spectral/hp elements is due to Robertson and Sherwin [97] using an arbitrary Lagrangian Eulerian (ALE) approach for the free water surface in a 2-D setting. Solving the FNPF equations is non-trivial, due to the need to evolve a set of highly nonlinear free surface boundary conditions in a robust way, together with the efficiency solution of a Laplace problems-possibly-in large ocean areas. For spectral/hp element models, Robertson and Sherwin identified a mesh asymmetry problem associated with triangulation of the fluid domain, which gives rise to real-valued eigenvalues and thus represented a severe instability problem. This was handled by adding a diffusive term in the kinematic free surface boundary condition proportional to the mesh asymmetry, but at the cost of reduced convergence rates. The problem of mesh asymmetry which destroys the dispersion relation was circumvented by Engsig-Karup et al. [98,99] by solving the FNPF equations in a - transformed domain [94] using a single layer of quadrilaterals in 2-D and prisms in 3-D. The FNPF model was shown to exhibit exponential convergence even for steep nonlinear stream function waves which most simpler wave models cannot handle. To stabilise the nonlinear simulations, it was found that the quartic nonlinear terms in the free surface equation need to be integrated exactly and for the steep waves a mild (1%) modal filter could be used to remove high frequency noise that can stem from aliasing in nonlinear terms for marginal resolution, and the gradient recovery steps that reconstruct the gradients of the solution (e.g., velocities) from the 0 C approximations.
Letting  denote the horizontal gradient operator, the velocity potential  satisfy the Laplace equa- with time-dependent free surface boundary conditions on = z   given in the Zakahrov [100] and zero normal flow on the bottom = z h   and on Here  is the free surface elevation, w the vertical velocity, h the still water depth and g the acceleration of gravity. The σ-transformed FNPF solver as presented in Ref. [99] has been implemented in the Nektar++ framework in order to allow for larger-scale modelling [101] . The solution is advanced explicitly in time by solving the free-surface equations. The computed is obtained by a 0 C gradient recovery as +1 n w  is needed to advance the free surface conditions another time step. Explicit timestepping schemes are effective for FNPF as the timestep restriction is not dependent on the horisontal mesh size, but only on the vertical resolution and water depth [98] . A σ-transformed FNPF solver is wellsuited for large-scale wave propagation simulations, since re-meshing is avoided. Also, an intrinsic property of FNPF solvers is the ability to get accurate kinematics in all of the fluid domain as a part of the solutions, which is relevant for wave-induced load predictions on marine structures. The - transformed FNPF solver can be used for computing e.g., shortwave disturbances in ports, see Fig. 16, and wave scattering from bottom mounted vertical cylinders, see Fig. 17.
The use of - transformed domains excludes any truncated bodies in the domain. In order to handle arbitrarily shaped bodies in the domain the mixed-Eulerian-Lagrangian (MEL) approach should be used and there are ongoing efforts to implement a SEM based on the MEL approach. In Ref. [102] a workaround on the mesh asymmetry problem associated with the MEL was presented. By using hybrid meshes consisting of a single layer of vertically aligned quads (in 2-D) at the free surface, the eigenvalues were shown to be purely imaginary for any triangulation of the inner field. When using higher-order local polynomial expansions as done in the spectral/hp element method, a local re-meshing (movement of quadrature points inside the local elements) as well as global re-meshing (movement of vertices) are needed in the MEL approach. An example of a spectral/hp element simulation using MEL for wave-body interaction are illustrated in Fig. 18. Here a solitary wave is passing over a horizontal submerged cylinder. Fig. 18 Solitary wave passing a submerged horizontal cylinder [102] 3.3.2 Shallow water wave equations While the application of spectral/hp element method for FNPF equations is emerging, in the field of depth-integrated shallow water long wave equations there is a wider use of spectral/hp elements methods. These equations can be derived from the FNPF equations by expanding the velocity potential in terms of the vertical coordinate and integrating the Laplace problem over the fluid depth. This removes the vertical dependence of the problem. Non-hydrostatic wave equations (such as Boussinesq-, Serre-or Green-Naghdi-type equations) are used for wave propagation and transformation in nearshore regions. The dispersive effects are included in the equations through higher-order mixed and spatial derivatives.
Eskilsson and Sherwin presented the first spectral/hp model for 1-D Boussinesq-type equa-tions [103] , and later in 2-D [104] . In the latter study was illustrated that the use of high-order schemes is computationally efficient compared to low-order methods for longtime integration. This is general knowledge, mathematically proven in Ref. [105], but the importance is accenturated in wave equation where the numerical dispersion error must be kept small compared to the physical dispersion terms. Additional spectral/hp studies of weakly dispersive Boussinesq equations are Refs. [106,107]. Engsig-Karup et al. [108][109][110] solved a set highly dispersive and nonlinear Boussinesq-type equations. More recently the interest has turned toward Green-Naghdi type equations due to their description of nonlinearity and a number of spectral/hp element methods have been put forward [111][112][113] . We note that all the above models are based on high-order DG methods.
The classical Boussinesq equations due to Peregrine [114] read where u is the horizontal velocity. There exists a pre-written solver for the Peregrine equations in Nektar++. The equations are solved in terms of conservative variables and the solution scheme is using the wave-continuity step as introduced in Ref. [104]. First the advection part is approximated and used in an intermediate step.  Fig. 19.
With regard to non-dispersive, hydrostatic wave equations the shallow water equations (SWE) are typically used in marine applications to predict tidal forcing but can also be used for other long wave phenomena such as tsunami and storm-surge predictions. For tsunami modelling hp-adaptive DG models have been presented in order to resolve the shock wave propagation in detail [116][117][118] . However, the main bulk of work on spectral/hp element methods for the SWE are linked to numerical weather prediction or ocean circulation where the SWE is used as a stepping stone towards solving the 3-D primitive equations [119][120][121][122][123][124][125] . Fig. 19 Higher harmonics generation on a semi-circular shoal [115] The SWE is solved in Nektar++ as a sub-model for the dispersive Boussinesq solver simply by ignoring the dispersive step. Recently a version of the SWE model capable of solving the flow on curved surfaces using Nektar++ was presented in Ref. [126]. The model uses the so-called method of moving frames [127] to express the SWE on curved and rotating geometries. The model was shown to exhibit exponential convergence and handled the standard test cases for spherical SWE, see Fig. 20 for the flow around an isolated mountain.

Future directions and perspectives
Significant progress has been made in theoretical and applied aspects of the spectral/hp element method over the past two decades which have enabled its application to a wide range of challenging hydrodynamics and broader industrial problems. However, there still remain a number of challenges [55] . In order to enable the spectral/hp element method to be used to address difficult, large-scale, high Reynolds number flow problems in complex geometries, we see a number of areas which still require improvement: (1) High-order mesh generation: Robustly gene-rating high-order meshes with CAD conforming representation of the underlying geometry remains one of the key challenges to industrial application. Fig. 20 Spherical shallow water flow over an isolated mountain [126] (2) One of the major benefits of the spectral/hp element method is the opportunities presented for optimising the computational discretisation in accordance with the physics being modelled through hp adaptivity. -p adaptivity and -r adaptivity have recently been explored [128,129] but further effort is needed in identifying the optimal refinement algorithms to choose to optimise the computational cost while also improving the numerical accuracy of solutions.
(3) The stabilisation techniques discussed in Section 1, such as SVV and projection-based filtering techniques, require further improvement through further calibration for marginal/under-resolved computations. As mentioned by Sagaut [37] , "some specific numerical stabilisation procedures can be defined, which tune the numerical dissipation in such a way that the results remain sensitive to subgrid modeling". In terms of the similarity between artificial dissipation and the direct energy cascade model, some numerical stabilisation techniques can be used to perform "no-model" large eddy simulations (iLES). However, Sagaut stated that "certain studies have shown that, for coarse grids, i.e., high values of the numerical cutoff length, increasing the order of accuracy of the upwind scheme can lead to a degradation of the results [130] ". The proposed "DGKernel", which emulates the upwind properties of a DG scheme, still requires more detailed calibration for a range of high-Reynolds fluid flows.
(4) Verification and Validation of spectral/hp element simulations for under-resolved computations: The use of stabilisation techniques, such as SVV, allows us to interpret the under-resolved numerical calculations as implicit large eddy simulation. However, we need to continue to investigate how the stabilisation methods and numerical properties interact to ensure the benefits of high-precision are not destroyed through artificial/ numerical pollution.
In the context of hydrodynamics applications, there is substantial opportunity for gaining an improved understanding of a wide range of physical mechanisms through the use of the spectral/hp element method. In particular: (1) Tip vortex modelling: As indicated in Ref. [7], without the accurate modelling of the 3-D boundary layer, the developing vortex remained challenging to compute accurately, even for the advanced RANS models correcting for the high degree of curvature in the flow. In addition, accurately capturing the lowpressure region within the vortex core and sustaining this low pressure even just one chord length downstream of the trailing edge is particularly challenging. As demonstrated [7] , the SVV-based iLES method has been shown to be potential alternative for computing complex unsteady vortex dominated flows. Simulating vortical flows using the spectral/hp element method with the concept of the SVV-based iLES can be regarded as an interesting alternative method in hydrodynamics.
(2) Implicit large eddy simulation: A notable effort has been made to refine the SVV-based strategy for implementing under-resolved simulations of high Reynolds fluid flows [38,44] . This strategy has a great potential to provide high fidelity iLES computations for a broad range of applications in hydrodynamics.
(4) Cavitating flow: Recently, discontinuous spectral/hp element method has been implemented for the efficient and accurate simulation of multiphase flows [137] . The compressible Navier-Stokes equations, coupled with a highly realistic equation of state, are solved by a DG spectral element method for flows with cavitation. In Nektar++, the DG method and the flux reconstruction approach have been used to solve the compressible Navier-Stokes equations. This therefore provides a basis to apply the spectral/hp element method in applications of cavitating flows.
(5) Free surface flows: As is well known free surface flows can be modelled as a depth average "shallow-water" approach or using ALE moving mesh techniques. Spectral/hp element method has been used to treat these two kinds of free surface flows [97,[138][139][140][141] . For the second free surface problem, discontinuous spectral/hp element methods hava great potential in applications [139-141, 143, 144] , especially when h/p-adaptivity is employed.