Finite Difference Computing with PDEs pp 353407  Cite as
Nonlinear Problems
Abstract
Algebraic equations
5.1 Introduction of Basic Concepts
5.1.1 Linear Versus Nonlinear Equations
Algebraic equations
We know how to solve a linear algebraic equation, x = −b ∕ a, but there are no general methods for finding the exact solutions of nonlinear algebraic equations, except for very special cases (quadratic equations constitute a primary example). A nonlinear algebraic equation may have no solution, one solution, or many solutions. The tools for solving nonlinear algebraic equations are iterative methods, where we construct a series of linear equations, which we know how to solve, and hope that the solutions of the linear equations converge to a solution of the nonlinear equation we want to solve. Typical methods for nonlinear algebraic equation equations are Newton’s method, the Bisection method, and the Secant method.
Differential equations
The unknown in a differential equation is a function and not a number. In a linear differential equation, all terms involving the unknown function are linear in the unknown function or its derivatives. Linear here means that the unknown function, or a derivative of it, is multiplied by a number or a known function. All other differential equations are nonlinear.
Mathematical proof of linearity
5.1.2 A Simple Model Problem

explicit time discretization methods (with no need to solve nonlinear algebraic equations)
 implicit Backward Euler time discretization, leading to nonlinear algebraic equations solved by

an exact analytical technique

Picard iteration based on manual linearization

a single Picard step

Newton’s method


implicit CrankNicolson time discretization and linearization via a geometric mean formula
5.1.3 Linearization by Explicit Time Discretization
Time discretization methods are divided into explicit and implicit methods. Explicit methods lead to a closedform formula for finding new values of the unknowns, while implicit methods give a linear or nonlinear system of equations that couples (all) the unknowns at a new time level. Here we shall demonstrate that explicit methods constitute an efficient way to deal with nonlinear differential equations.
5.1.4 Exact Solution of Nonlinear Algebraic Equations
Now we encounter a fundamental challenge with nonlinear algebraic equations: the equation may have more than one solution. How do we pick the right solution? This is in general a hard problem. In the present simple case, however, we can analyze the roots mathematically and provide an answer. The idea is to expand the roots in a series in \(\Delta t\) and truncate after the linear term since the Backward Euler scheme will introduce an error proportional to \(\Delta t\) anyway. Using sympy, we find the following Taylor series expansions of the roots:
We see that the r1 root, corresponding to a minus sign in front of the square root in (5.4), behaves as \(1/\Delta t\) and will therefore blow up as \(\Delta t\rightarrow 0\)! Since we know that u takes on finite values, actually it is less than or equal to 1, only the r2 root is of relevance in this case: as \(\Delta t\rightarrow 0\), \(u\rightarrow u^{(1)}\), which is the expected result.
For those who are not well experienced with approximating mathematical formulas by series expansion, an alternative method of investigation is simply to compute the limits of the two roots as \(\Delta t\rightarrow 0\) and see if a limit appears unreasonable:
5.1.5 Linearization
When the time integration of an ODE results in a nonlinear algebraic equation, we must normally find its solution by defining a sequence of linear equations and hope that the solutions of these linear equations converge to the desired solution of the nonlinear algebraic equation. Usually, this means solving the linear equation repeatedly in an iterative fashion. Alternatively, the nonlinear equation can sometimes be approximated by one linear equation, and consequently there is no need for iteration.
Constructing a linear equation from a nonlinear one requires linearization of each nonlinear term. This can be done manually as in Picard iteration, or fully algorithmically as in Newton’s method. Examples will best illustrate how to linearize nonlinear problems.
5.1.6 Picard Iteration
The idea of turning a nonlinear equation into a linear one by using an approximation \(u^{}\) of u in nonlinear terms is a widely used approach that goes under many names: fixedpoint iteration, the method of successive substitutions, nonlinear Richardson iteration, and Picard iteration. We will stick to the latter name.
However, we will normally apply a mathematical notation in our final formulas that is as close as possible to what we aim to write in a computer code and then it becomes natural to use u and \(u^{}\) instead of \(u^{k+1}\) and u ^{ k } or \(u^{n,k+1}\) and u ^{n,k}.
Stopping criteria
A single Picard iteration
We shall later refer to the strategy of taking one Picard step, or equivalently, linearizing terms with use of the solution at the previous time step, as the Picard1 method. It is a widely used approach in science and technology, but with some limitations if \(\Delta t\) is not sufficiently small (as will be illustrated later).
Notice
Equation (5.5) does not correspond to a ‘‘pure’’ finite difference method where the equation is sampled at a point and derivatives replaced by differences (because the \(u^{n1}\) term on the righthand side must then be u ^{ n }). The best interpretation of the scheme (5.5) is a Backward Euler difference combined with a single (perhaps insufficient) Picard iteration at each time level, with the value at the previous time level as start for the Picard iteration.
5.1.7 Linearization by a Geometric Mean
The geometric mean approximation is often very effective for linearizing quadratic nonlinearities. Both the arithmetic and geometric mean approximations have truncation errors of order \(\Delta t^{2}\) and are therefore compatible with the truncation error \(\mathcal{O}(\Delta t^{2})\) of the centered difference approximation for \(u^{\prime}\) in the CrankNicolson method.
Remark
If we use an arithmetic instead of a geometric mean for the nonlinear term in (5.6), we end up with a nonlinear term \((u^{n+1})^{2}\). This term can be linearized as \(u^{}u^{n+1}\) in a Picard iteration approach and in particular as \(u^{n}u^{n+1}\) in a Picard1 iteration approach. The latter gives a scheme almost identical to the one arising from a geometric mean (the difference in \(u^{n+1}\) being \(\frac{1}{4}\Delta tu^{n}(u^{n+1}u^{n})\approx\frac{1}{4}\Delta t^{2}u^{\prime}u\), i.e., a difference of size \(\Delta t^{2}\)).
5.1.8 Newton’s Method
It can be shown that the error in iteration k + 1 of Newton’s method is proportional to the square of the error in iteration k, a result referred to as quadratic convergence. This means that for small errors the method converges very fast, and in particular much faster than Picard iteration and other iteration methods. (The proof of this result is found in most textbooks on numerical analysis.) However, the quadratic convergence appears only if u ^{ k } is sufficiently close to the solution. Further away from the solution the method can easily converge very slowly or diverge. The reader is encouraged to do Exercise 5.3 to get a better understanding for the behavior of the method.
5.1.9 Relaxation
5.1.10 Implementation and Experiments
The program logistic.py contains implementations of all the methods described above. Below is an extract of the file showing how the Picard and Newton methods are implemented for a Backward Euler discretization of the logistic equation.
The CrankNicolson method utilizing a linearization based on the geometric mean gives a simpler algorithm:
The BE exact curve corresponds to using the exact solution of the quadratic equation at each time level, so this curve is only affected by the Backward Euler time discretization. The CN gm curve corresponds to the theoretically more accurate CrankNicolson discretization, combined with a geometric mean for linearization. This curve appears more accurate, especially if we take the plot in the lower right with a small \(\Delta t\) and an appropriately small ϵ_{ r } value as the exact curve.
However, introducing relaxation and a parameter ω = 0.8 immediately reduces the average of 32 to 7, indicating that for the large \(\Delta t=0.9\), Picard iteration takes too long steps. An approximately optimal value for ω in this case is 0.5, which results in an average of only 2 iterations! An even more dramatic impact of ω appears when \(\Delta t=1\): Picard iteration does not convergence in 1000 iterations, but ω = 0.5 again brings the average number of iterations down to 2.
Remark
The simple CrankNicolson method with a geometric mean for the quadratic nonlinearity gives visually more accurate solutions than the Backward Euler discretization. Even with a tolerance of \(\epsilon_{r}=10^{3}\), all the methods for treating the nonlinearities in the Backward Euler discretization give graphs that cannot be distinguished. So for accuracy in this problem, the time discretization is much more crucial than ϵ_{ r }. Ideally, one should estimate the error in the time discretization, as the solution progresses, and set ϵ_{ r } accordingly.
5.1.11 Generalization to a General Nonlinear ODE
Explicit time discretization
Explicit ODE methods like the Forward Euler scheme, RungeKutta methods and AdamsBashforth methods all evaluate f at time levels where u is already computed, so nonlinearities in f do not pose any difficulties.
Backward Euler discretization
Explicit vs implicit treatment of nonlinear terms
Evaluating f for a known \(u^{}\) is referred to as explicit treatment of f, while if \(f(u,t)\) has some structure, say \(f(u,t)=u^{3}\), parts of f can involve the unknown u, as in the manual linearization \((u^{})^{2}u\), and then the treatment of f is ‘‘more implicit’’ and ‘‘less explicit’’. This terminology is inspired by time discretization of \(u^{\prime}=f(u,t)\), where evaluating f for known u values gives explicit schemes, while treating f or parts of f implicitly, makes f contribute to the unknown terms in the equation at the new time level.
Explicit treatment of f usually means stricter conditions on \(\Delta t\) to achieve stability of time discretization schemes. The same applies to iteration techniques for nonlinear algebraic equations: the ‘‘less’’ we linearize f (i.e., the more we keep of u in the original formula), the faster the convergence may be.
We may say that \(f(u,t)=u^{3}\) is treated explicitly if we evaluate f as \((u^{})^{3}\), partially implicit if we linearize as \((u^{})^{2}u\) and fully implicit if we represent f by u ^{3}. (Of course, the fully implicit representation will require further linearization, but with \(f(u,t)=u^{2}\) a fully implicit treatment is possible if the resulting quadratic equation is solved with a formula.)
For the ODE \(u^{\prime}=u^{3}\) with \(f(u,t)=u^{3}\) and coarse time resolution \(\Delta t=0.4\), Picard iteration with \((u^{})^{2}u\) requires 8 iterations with \(\epsilon_{r}=10^{3}\) for the first time step, while \((u^{})^{3}\) leads to 22 iterations. After about 10 time steps both approaches are down to about 2 iterations per time step, but this example shows a potential of treating f more implicitly.
A trick to treat f implicitly in Picard iteration is to evaluate it as \(f(u^{},t)u/u^{}\). For a polynomial f, \(f(u,t)=u^{m}\), this corresponds to \((u^{})^{m}u/u^{}=(u^{})^{m1}u\). Sometimes this more implicit treatment has no effect, as with \(f(u,t)=\exp(u)\) and \(f(u,t)=\ln(1+u)\), but with \(f(u,t)=\sin(2(u+1))\), the \(f(u^{},t)u/u^{}\) trick leads to 7, 9, and 11 iterations during the first three steps, while \(f(u^{},t)\) demands 17, 21, and 20 iterations. (Experiments can be done with the code ODE_Picard_tricks.py .)
CrankNicolson discretization
5.1.12 Systems of ODEs
Example
5.2 Systems of Nonlinear Algebraic Equations
We shall next explain how Picard iteration and Newton’s method can be applied to systems like \(F(u)=0\) and \(A(u)u=b(u)\). The exposition has a focus on ideas and practical computations. More theoretical considerations, including quite general results on convergence properties of these methods, can be found in Kelley [8].
5.2.1 Picard Iteration
Algorithm for relaxed Picard iteration
 1.
solve \(A(u^{})u^{*}=b(u^{})\) with respect to \(u^{*}\)
 2.
\(u=\omega u^{*}+(1\omega)u^{}\)
 3.
\(u^{}\ \leftarrow\ u\)
‘‘Until convergence’’ means that the iteration is stopped when the change in the unknown, \(uu^{}\), or the residual \(A(u)ub\), is sufficiently small, see Sect. 5.2.3 for more details.
5.2.2 Newton’s Method
Algorithm for Newton’s method
 1.
solve \(J\delta u=F(u^{})\) with respect to \(\delta u\)
 2.
\(u=u^{}+\omega\delta u\)
 3.
\(u^{}\ \leftarrow\ u\)
Combined algorithm for Picard and Newton iteration
 1.
solve \((A+\gamma(A^{\prime}(u^{})u^{}+b^{\prime}(u^{})))\delta u=A(u^{})u^{}+b(u^{})\) with respect to \(\delta u\)
 2.
\(u=u^{}+\omega\delta u\)
 3.
\(u^{}\ \leftarrow\ u\)
5.2.3 Stopping Criteria

Absolute change in solution: \(uu^{}\leq\epsilon_{u}\)

Relative change in solution: \(uu^{}\leq\epsilon_{u}u_{0}\), where u _{0} denotes the start value of \(u^{}\) in the iteration

Absolute residual: \(F(u)\leq\epsilon_{r}\)

Relative residual: \(F(u)\leq\epsilon_{r}F(u_{0})\)
5.2.4 Example: A Nonlinear ODE Model from Epidemiology
Implicit time discretization
A Picard iteration
Newton’s method
Remark
For this particular system of ODEs, explicit time integration methods work very well. Even a Forward Euler scheme is fine, but (as also experienced more generally) the 4th order RungeKutta method is an excellent balance between high accuracy, high efficiency, and simplicity.
5.3 Linearization at the Differential Equation Level
In the present section, our aim is to discretize this problem in time and then present techniques for linearizing the timediscrete PDE problem ‘‘at the PDE level’’ such that we transform the nonlinear stationary PDE problem at each time level into a sequence of linear PDE problems, which can be solved using any method for linear PDEs. This strategy avoids the solution of systems of nonlinear algebraic equations. In Sect. 5.4 we shall take the opposite (and more common) approach: discretize the nonlinear problem in time and space first, and then solve the resulting nonlinear algebraic equations at each time level by the methods of Sect. 5.2. Very often, the two approaches are mathematically identical, so there is no preference from a computational efficiency point of view. The details of the ideas sketched above will hopefully become clear through the forthcoming examples.
5.3.1 Explicit Time Integration
The disadvantage with this discretization is the strict stability criterion \(\Delta t\leq h^{2}/(6\max\alpha)\) for the case f = 0 and a standard 2ndorder finite difference discretization in 3D space with mesh cell sizes \(h=\Delta x=\Delta y=\Delta z\).
5.3.2 Backward Euler Scheme and Picard Iteration
Remark on notation
The previous derivations of the numerical scheme for time discretizations of PDEs have, strictly speaking, a somewhat sloppy notation, but it is much used and convenient to read. A more precise notation must distinguish clearly between the exact solution of the PDE problem, here denoted \(u_{\mbox{\footnotesize e}}(\boldsymbol{x},t)\), and the exact solution of the spatial problem, arising after time discretization at each time level, where (5.33) is an example. The latter is here represented as \(u^{n}(\boldsymbol{x})\) and is an approximation to \(u_{\mbox{\footnotesize e}}(\boldsymbol{x},t_{n})\). Then we have another approximation \(u^{n,k}(\boldsymbol{x})\) to \(u^{n}(\boldsymbol{x})\) when solving the nonlinear PDE problem for u ^{ n } by iteration methods, as in (5.34).
In our notation, u is a synonym for \(u^{n,k+1}\) and \(u^{(1)}\) is a synonym for \(u^{n1}\), inspired by what are natural variable names in a code. We will usually state the PDE problem in terms of u and quickly redefine the symbol u to mean the numerical approximation, while \(u_{\mbox{\footnotesize e}}\) is not explicitly introduced unless we need to talk about the exact solution and the approximate solution at the same time.
5.3.3 Backward Euler Scheme and Newton’s Method
At time level n, we have to solve the stationary PDE (5.33). In the previous section, we saw how this can be done with Picard iterations. Another alternative is to apply the idea of Newton’s method in a clever way. Normally, Newton’s method is defined for systems of algebraic equations, but the idea of the method can be applied at the PDE level too.
Linearization via Taylor expansions
Observations
The notational form \(\delta F=F\) resembles the Newton system \(J\delta u=F\) for systems of algebraic equations, with \(\delta F\) as \(J\delta u\). The unknown vector in a linear system of algebraic equations enters the system as a linear operator in terms of a matrixvector product (\(J\delta u\)), while at the PDE level we have a linear differential operator instead (\(\delta F\)).
Similarity with Picard iteration
Implementation
Derivation with alternative notation
5.3.4 CrankNicolson Discretization
A big question is whether there are significant differences in accuracy between taking the products of arithmetic means or taking the arithmetic mean of products. Exercise 5.6 investigates this question, and the answer is that the approximation is \(\mathcal{O}(\Delta t^{2})\) in both cases.
5.4 1D Stationary Nonlinear Differential Equations
Section 5.3 presented methods for linearizing timediscrete PDEs directly prior to discretization in space. We can alternatively carry out the discretization in space of the timediscrete nonlinear PDE problem and get a system of nonlinear algebraic equations, which can be solved by Picard iteration or Newton’s method as presented in Sect. 5.2. This latter approach will now be described in detail.
5.4.1 Finite Difference Discretization
5.4.2 Solution of Algebraic Equations
The structure of the equation system
Regarding the last equation, its form depends on whether we include the Dirichlet condition \(u(L)=D\), meaning \(u_{N_{x}}=D\), in the nonlinear algebraic equation system or not. Suppose we choose \((u_{0},u_{1},\ldots,u_{N_{x}1})\) as unknowns, later referred to as systems without Dirichlet conditions. The last equation corresponds to \(i=N_{x}1\). It involves the boundary value \(u_{N_{x}}\), which is substituted by D. If the unknown vector includes the boundary value, \((u_{0},u_{1},\ldots,u_{N_{x}})\), later referred to as system including Dirichlet conditions, the equation for \(i=N_{x}1\) just involves the unknown \(u_{N_{x}}\), and the final equation becomes \(u_{N_{x}}=D\), corresponding to \(A_{i,i}=1\) and \(b_{i}=D\) for \(i=N_{x}\).
Picard iteration
The obvious Picard iteration scheme is to use previously computed values of u _{ i } in A(u) and b(u), as described more in detail in Sect. 5.2. With the notation \(u^{}\) for the most recently computed value of u, we have the system \(F(u)\approx\hat{F}(u)=A(u^{})ub(u^{})\), with \(F=(F_{0},F_{1},\ldots,F_{m})\), \(u=(u_{0},u_{1},\ldots,u_{m})\). The index m is N _{ x } if the system includes the Dirichlet condition as a separate equation and \(N_{x}1\) otherwise. The matrix \(A(u^{})\) is tridiagonal, so the solution procedure is to fill a tridiagonal matrix data structure and the righthand side vector with the right numbers and call a Gaussian elimination routine for tridiagonal linear systems.
Mesh with two cells
It helps on the understanding of the details to write out all the mathematics in a specific case with a small mesh, say just two cells (\(N_{x}=2\)). We use \(u^{}_{i}\) for the ith component in \(u^{}\).
Newton’s method
We have seen, and can see from the present example, that the linear system in Newton’s method contains all the terms present in the system that arises in the Picard iteration method. The extra terms in Newton’s method can be multiplied by a factor such that it is easy to program one linear system and set this factor to 0 or 1 to generate the Picard or Newton system.
5.5 MultiDimensional Nonlinear PDE Problems
The fundamental ideas in the derivation of F _{ i } and J _{i,j} in the 1D model problem are easily generalized to multidimensional problems. Nevertheless, the expressions involved are slightly different, with derivatives in x replaced by \(\nabla\), so we present some examples below in detail.
5.5.1 Finite Difference Discretization
Picard iteration
The most recently computed values \(u^{}\) of u ^{ n } can be used in α and f for a Picard iteration, or equivalently, we solve \(A(u^{})u=b(u^{})\). The result is a linear system of the same type as arising from \(u_{t}=\nabla\cdot(\alpha(\boldsymbol{x})\nabla u)+f(\boldsymbol{x},t)\).
Newton’s method
5.5.2 Continuation Methods
Picard iteration or Newton’s method may diverge when solving PDEs with severe nonlinearities. Relaxation with ω < 1 may help, but in highly nonlinear problems it can be necessary to introduce a continuation parameter Λ in the problem: Λ = 0 gives a version of the problem that is easy to solve, while Λ = 1 is the target problem. The idea is then to increase Λ in steps, \(\Lambda_{0}=0,\Lambda_{1}<\cdots<\Lambda_{n}=1\), and use the solution from the problem with \(\Lambda_{i1}\) as initial guess for the iterations in the problem corresponding to Λ_{ i }.
5.6 Operator Splitting Methods
Operator splitting is a natural and old idea. When a PDE or system of PDEs contains different terms expressing different physics, it is natural to use different numerical methods for different physical processes. This can optimize and simplify the overall solution process. The idea was especially popularized in the context of the NavierStokes equations and reactiondiffusion PDEs. Common names for the technique are operator splitting, fractional step methods, and splitstep methods. We shall stick to the former name. In the context of nonlinear differential equations, operator splitting can be used to isolate nonlinear terms and simplify the solution methods.
A related technique, often known as dimensional splitting or alternating direction implicit (ADI) methods, is to split the spatial dimensions and solve a 2D or 3D problem as two or three consecutive 1D problems, but this type of splitting is not to be further considered here.
5.6.1 Ordinary Operator Splitting for ODEs
The technique is exact if the ODEs are linear. For nonlinear ODEs it is only an approximate method with error \(\Delta t\). The technique can be extended to an arbitrary number of steps; i.e., we may split the PDE system into any number of subsystems. Examples will illuminate this principle.
5.6.2 Strang Splitting for ODEs
There is no use in combining higherorder methods with ordinary splitting since the error due to splitting is \(\mathcal{O}(\Delta t)\), but for Strang splitting it makes sense to use schemes of order \(\mathcal{O}(\Delta t^{2})\).
5.6.3 Example: Logistic Growth
Splitting techniques
Verbose implementation
The following function computes four solutions arising from the Forward Euler method, ordinary splitting, Strang splitting, as well as Strang splitting with exact treatment of \(u^{\prime}=f_{0}(u)\):
Compact implementation
We have used quite many lines for the steps in the splitting methods. Many will prefer to condense the code a bit, as done here:
Results
As technique for solving nonlinear ODEs, we realize that the present case study is not particularly promising, as the Forward Euler method both linearizes the original problem and provides a solution that is much more accurate than any of the splitting techniques. In complicated multiphysics settings, on the other hand, splitting may be the only feasible way to go, and sometimes you really need to apply different numerics to different parts of a PDE problem. But in very simple problems, like the logistic ODE, splitting is just an inferior technique. Still, the logistic ODE is ideal for introducing all the mathematical details and for investigating the behavior.
5.6.4 ReactionDiffusion Equation
 1.
Solve the diffusion problem for one time step as usual.
 2.
Solve the reaction ODEs at each mesh point in \([t_{n},t_{n}+\Delta t]\), using the diffusion solution in 1. as initial condition. The solution of the ODEs constitutes the solution of the original problem at the end of each time step.
5.6.5 Example: ReactionDiffusion with Linear Reaction Term
The methods above may be explored in detail through a specific computational example in which we compute the convergence rates associated with four different solution approaches for the reactiondiffusion equation with a linear reaction term, i.e. \(f(u)=bu\). The methods comprise solving without splitting (just straight Forward Euler), ordinary splitting, first order Strang splitting, and second order Strang splitting. In all four methods, a standard centered difference approximation is used for the spatial second derivative. The methods share the error model \(E=Ch^{r}\), while differing in the step h (being either \(\Delta x^{2}\) or \(\Delta x\)) and the convergence rate r (being either 1 or 2).
All code commented below is found in the file split_diffu_react.py . When executed, a function convergence_rates is called, from which all convergence rate computations are handled:
Now, with respect to the error (\(E=Ch^{r}\)), the Forward Euler scheme, the ordinary splitting scheme and first order Strang splitting scheme are all first order (r = 1), with a step \(h=\Delta x^{2}=K^{1}\Delta t\), where K is some constant. This implies that the ratio \(\frac{\Delta t}{\Delta x^{2}}\) must be held constant during convergence rate calculations. Furthermore, the Fourier number \(F=\frac{\alpha\Delta t}{\Delta x^{2}}\) is upwards limited to F = 0.5, being the stability limit with explicit schemes. Thus, in these cases, we use the fixed value of F and a given (but changing) spatial resolution \(\Delta x\) to compute the corresponding value of \(\Delta t\) according to the expression for F. This assures that \(\frac{\Delta t}{\Delta x^{2}}\) is kept constant. The loop in convergence_rates runs over a chosen set of grid points (Nx_values) which gives a doubling of spatial resolution with each iteration (\(\Delta x\) is halved).
For the second order Strang splitting scheme, we have r = 2 and a step \(h=\Delta x=K^{1}\Delta t\), where K again is some constant. In this case, it is thus the ratio \(\frac{\Delta t}{\Delta x}\) that must be held constant during the convergence rate calculations. From the expression for F, it is clear then that F must change with each halving of \(\Delta x\). In fact, if F is doubled each time \(\Delta x\) is halved, the ratio \(\frac{\Delta t}{\Delta x}\) will be constant (this follows, e.g., from the expression for F). This is utilized in our code.
A solver diffusion_theta is used in each of the four solution approaches:
For the no splitting approach with Forward Euler in time, this solver handles both the diffusion and the reaction term. When splitting, diffusion_theta takes care of the diffusion term only, while the reaction term is handled either by a Forward Euler scheme in reaction_FE, or by a second order AdamsBashforth scheme from Odespy. The reaction_FE function covers one complete time step dt during ordinary splitting, while Strang splitting (both first and second order) applies it with dt/2 twice during each time step dt. Since the reaction term typically represents a much faster process than the diffusion term, a further refinement of the time step is made possible in reaction_FE. It was implemented as
With the ordinary splitting approach, each time step dt is covered twice. First computing the impact of the reaction term, then the contribution from the diffusion term:
For the two Strang splitting approaches, each time step dt is handled by first computing the reaction step for (the first) dt/2, followed by a diffusion step dt, before the reaction step is treated once again for (the remaining) dt/2. Since first order Strang splitting is no better than first order accurate, both the reaction and diffusion steps are computed explicitly. The solver was implemented as
The second order version of the Strang splitting approach utilizes a second order AdamsBashforth solver for the reaction part and a CrankNicolson scheme for the diffusion part. The solver has the same structure as the one for first order Strang splitting and was implemented as
When executing split_diffu_react.py, we find that the estimated convergence rates are as expected. The second order Strang splitting gives the least error (about 4e^{−5}) and has second order convergence (r = 2), while the remaining three approaches have first order convergence (r = 1).
5.6.6 Analysis of the Splitting Method
Let us address a linear PDE problem for which we can develop analytical solutions of the discrete equations, with and without splitting, and discuss these. Choosing \(f(u)=\beta u\) for a constant β gives a linear problem. We use the Forward Euler method for both the PDE and ODE problems.
5.7 Exercises
Problem 5.1 (Determine if equations are nonlinear or not)
 1.
\(mu^{\prime\prime}+\betau^{\prime}u^{\prime}+cu=F(t)\)
 2.
\(u_{t}=\alpha u_{xx}\)
 3.
\(u_{tt}=c^{2}\nabla^{2}u\)
 4.
\(u_{t}=\nabla\cdot(\alpha(u)\nabla u)+f(x,y)\)
 5.
\(u_{t}+f(u)_{x}=0\)
 6.
\(\boldsymbol{u}_{t}+\boldsymbol{u}\cdot\nabla\boldsymbol{u}=\nabla p+r\nabla^{2}\boldsymbol{u}\), \(\nabla\cdot\boldsymbol{u}=0\) (u is a vector field)
 7.
\(u^{\prime}=f(u,t)\)
 8.
\(\nabla^{2}u=\lambda e^{u}\)
Filename: nonlinear_vs_linear.
Problem 5.2 (Derive and investigate a generalized logistic model)
 a)
Formulate a Forward Euler, Backward Euler, and a CrankNicolson scheme for (5.82).
Hint
Use a geometric mean approximation in the CrankNicolson scheme: \([a(u)u]^{n+1/2}\approx a(u^{n})u^{n+1}\).
 a)
Formulate Picard and Newton iteration for the Backward Euler scheme in a).
 b)
Implement the numerical solution methods from a) and b). Use logistic.py to compare the case p = 1 and the choice (5.83).
 c)
Implement unit tests that check the asymptotic limit of the solutions: \(u\rightarrow M\) as \(t\rightarrow\infty\).
Hint
You need to experiment to find what ‘‘infinite time’’ is (increases substantially with p) and what the appropriate tolerance is for testing the asymptotic limit.
 e)
Perform experiments with Newton and Picard iteration for the model (5.83). See how sensitive the number of iterations is to \(\Delta t\) and p.
Filename: logistic_p.
Problem 5.3 (Experience the behavior of Newton’s method)
The program Newton_demo.py illustrates graphically each step in Newton’s method and is run like
Use this program to investigate potential problems with Newton’s method when solving \(e^{0.5x^{2}}\cos(\pi x)=0\). Try a starting point \(x_{0}=0.8\) and \(x_{0}=0.85\) and watch the different behavior. Just run
and repeat with 0.85 replaced by 0.8.
Exercise 5.4 (Compute the Jacobian of a 2 × 2 system)
Write up the system (5.18)–(5.19) in the form \(F(u)=0\), \(F=(F_{0},F_{1})\), \(u=(u_{0},u_{1})\), and compute the Jacobian \(J_{i,j}=\partial F_{i}/\partial u_{j}\).
Problem 5.5 (Solve nonlinear equations arising from a vibration ODE)
 a)
Rewrite the equation for u as a system of two firstorder ODEs, and discretize this system by a CrankNicolson (centered difference) method. With \(v=u^{\prime}\), we get a nonlinear term \(v^{n+\frac{1}{2}}v^{n+\frac{1}{2}}\). Use a geometric average for \(v^{n+\frac{1}{2}}\).
 b)
Formulate a Picard iteration method to solve the system of nonlinear algebraic equations.
 c)
Explain how to apply Newton’s method to solve the nonlinear equations at each time level. Derive expressions for the Jacobian and the righthand side in each Newton iteration.
Filename: nonlin_vib.
Exercise 5.6 (Find the truncation error of arithmetic mean of products)
Hint
You may explore sympy for carrying out the tedious calculations. A general Taylor series expansion of \(P(t+\frac{1}{2}\Delta t)\) around t involving just a general function P(t) can be created as follows:
The error of the arithmetic mean, \(\frac{1}{2}(P(\frac{1}{2}\Delta t)+P(\frac{1}{2}\Delta t))\) for t = 0 is then
Use these examples to investigate the error of (5.85) and (5.86) for n = 0. (Choosing n = 0 is necessary for not making the expressions too complicated for sympy, but there is of course no lack of generality by using n = 0 rather than an arbitrary n  the main point is the product and addition of Taylor series.)
Filename: product_arith_mean.
Problem 5.7 (Newton’s method for linear problems)
Suppose we have a linear system \(F(u)=Aub=0\). Apply Newton’s method to this system, and show that the method converges in one iteration.
Filename: Newton_linear.
Problem 5.8 (Discretize a 1D problem with a nonlinear coefficient)
Discretize (5.87) by a centered finite difference method on a uniform mesh.
Filename: nonlin_1D_coeff_discretize.
Problem 5.9 (Linearize a 1D problem with a nonlinear coefficient)
 a)
Construct a Picard iteration method for (5.88) without discretizing in space.
 b)
Apply Newton’s method to (5.88) without discretizing in space.
 c)
Discretize (5.88) by a centered finite difference scheme. Construct a Picard method for the resulting system of nonlinear algebraic equations.
 d)
Discretize (5.88) by a centered finite difference scheme. Define the system of nonlinear algebraic equations, calculate the Jacobian, and set up Newton’s method for solving the system.
Filename: nonlin_1D_coeff_linearize.
Problem 5.10 (Finite differences for the 1D Bratu problem)
 a)
Discretize (5.89) by a centered finite difference method.
 b)
Set up the nonlinear equations \(F_{i}(u_{0},u_{1},\ldots,u_{N_{x}})=0\) from a). Calculate the associated Jacobian.
 c)
Implement a solver that can compute u(x) using Newton’s method. Plot the error as a function of x in each iteration.
 d)
Investigate whether Newton’s method gives secondorder convergence by computing \(u_{\mbox{\footnotesize e}}u/u_{\mbox{\footnotesize e}}u^{}^{2}\) in each iteration, where u is solution in the current iteration and \(u^{}\) is the solution in the previous iteration.
Filename: nonlin_1D_Bratu_fd.
Problem 5.11 (Discretize a nonlinear 1D heat conduction PDE by finite differences)
 a)
Discretize this PDE in time using either a Backward Euler or CrankNicolson scheme.
 b)
Formulate a Picard iteration method for the timediscrete problem (i.e., an iteration method before discretizing in space).
 c)
Formulate a Newton method for the timediscrete problem in b).
 d)
Discretize the PDE by a finite difference method in space. Derive the matrix and righthand side of a Picard iteration method applied to the spacetime discretized PDE.
 e)
Derive the matrix and righthand side of a Newton method applied to the discretized PDE in d).
Filename: nonlin_1D_heat_FD.
Problem 5.12 (Differentiate a highly nonlinear term)
Filename: nonlin_differentiate.
Exercise 5.13 (CrankNicolson for a nonlinear 3D diffusion equation)
Redo Sect. 5.5.1 when a CrankNicolson scheme is used to discretize the equations in time and the problem is formulated for three spatial dimensions.
Hint
Express the Jacobian as \(J_{i,j,k,r,s,t}=\partial F_{i,j,k}/\partial u_{r,s,t}\) and observe, as in the 2D case, that J _{i,j,k,r,s,t} is very sparse: \(J_{i,j,k,r,s,t}\neq 0\) only for r = i ± i, s = j ± 1, and t = k ± 1 as well as r = i, s = j, and t = k.
Filename: nonlin_heat_FD_CN_2D.
Problem 5.14 (Find the sparsity of the Jacobian)
Consider a typical nonlinear Laplace term like \(\nabla\cdot\alpha(u)\nabla u\) discretized by centered finite differences. Explain why the Jacobian corresponding to this term has the same sparsity pattern as the matrix associated with the corresponding linear term \(\alpha\nabla^{2}u\).
Hint
Set up the unknowns that enter the difference equation at a point \((i,j)\) in 2D or \((i,j,k)\) in 3D, and identify the nonzero entries of the Jacobian that can arise from such a type of difference equation.
Filename: nonlin_sparsity_Jacobian.
Problem 5.15 (Investigate a 1D problem with a continuation method)
 a)
Formulate a Picard iteration method directly for the differential equation problem.
 b)
Perform a finite difference discretization of the problem in each Picard iteration. Implement a solver that can compute u on a mesh. Verify that the solver gives an exact solution for n = 1 on a uniform mesh regardless of the cell size.
 c)
Given a sequence of decreasing n values, solve the problem for each n using the solution for the previous n as initial guess for the Picard iteration. This is called a continuation method. Experiment with \(n=(1,0.6,0.2)\) and \(n=(1,0.9,0.8,\ldots,0.2)\) and make a table of the number of Picard iterations versus n.
 d)
Derive a Newton method at the differential equation level and discretize the resulting linear equations in each Newton iteration with the finite difference method.
 e)
Investigate if Newton’s method has better convergence properties than Picard iteration, both in combination with a continuation method.
Copyright information
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made. The images or other third party material in this chapter are included in the chapter’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.