Basic Fourier Series

Chapter

Abstract

We will now study the basic rules of Fourier Series. You can study these basic principles for years in order to completely understand some of the many subtleties. The concentration of this book will be to quickly understand Fourier Series at a level which will allow to study their many applications. The reader is then encouraged to look further for a more in-depth understanding of the topics from the many other resources (Benedetto, Harmonic analysis and applications, CRC Press, 1996, [1]; Bracewell, The Fourier Transform and its applications, 1999, [3]; Cheney, Approximation theory, Chelsea Publishing Company, New York, [5]; Rudin, Real and complex analysis, McGraw-Hill, 1974, [17]; Tolstov, Fourier Series, Dover, London, 1962, [20]).

We will now study the basic rules of Fourier Series. You can study these basic principles for years in order to completely understand some of the many subtleties. The concentration of this book will be to quickly understand Fourier Series at a level which will allow to study their many applications. The reader is then encouraged to look further for a more in-depth understanding of the topics from the many other resources [1, 3, 5, 17, 20].

2.1 Fourier Series on \(L^2[a, b]\)

We begin with the most basic Fourier Series and outline how we can adjust them to other intervals. We will then outline the mathematics which forms the basis for these claims, and the many implications and structure which gives background to the study of Fourier Series.

We must begin with the basic definition of \(L^2[a, b]\)

Definition 2.1.1

The set of functions \(f(t): [a, b] \rightarrow R \) whose squared integral is finite, or \(\int _a^b |f(t)|^2 < \infty \) is referred to as \(L^2[a, b]\), or the square integrable functions on [ab].

Thus, the square integrable  functions form the space  of functions which we wish to study. We need to establish that if two functions \(f(t), g(t) \in L^2[a, b]\), their  linear sums \(c_1 f(t) + c_2 g(t)\) are also in this space for any constants \(c_1\) and \(c_2\). To see this for real-valued functions, we simply write
$$\begin{aligned} \Vert c_1 f(t) + c_2 g(t) \Vert _2^2= & {} \int _a^b ( c_1 f(t) + c_2 g(t) )^2 dt \nonumber \\= & {} \int _a^b c_1^2 f(t)^2 + c_1 c_2 f(t) g(t) + c_2^2 g(t)^2 dt \nonumber \\= & {} c_1^2 \int _a^b f(t)^2 dt + c_2^2 \int _a^b g(t)^2 \nonumber \\+ & {} c_1 c_2 \int _a^b f(t) g(t) dt. \end{aligned}$$
(2.1)
The first and second integrals in (2.1) are finite because f(t) and \(g(t) \in L^2[a, b]\). The third integral is finite because of the Cauchy–Schwartz theorem which states that
$$ |\int _a^b f(t) g(t) dt|^2 \le \Vert f(t) \Vert _2^2 \Vert g(t) \Vert _2^2 . $$
Thus, it is a vector space. The proof when the functions are complex is left as an exercise.

We now state the most basic theorem of this section.

Theorem 2.1.1

Let f(t) be any function in \(L^2[-\pi ,\pi ]\). Then, we can represent f(t) in a series as
$$ f(t) = \frac{a_0}{2} + \sum _{k=1}^\infty a_k\cos (kt) + b_k \sin (kt) $$
where
$$ a_k = \frac{1}{\pi } \int _{-\pi }^\pi f(t)\cos (kt) dt , \text{ and } b_k = \frac{1}{\pi } \int _{-\pi }^\pi f(t) \sin ( kt ) dt . $$

Thus, you can represent any function  in \(L^2[-\pi ,\pi ]\) as a sum of sines and cosines. We can even state more than this:

Theorem 2.1.2

Let f(t) be any function in \(L^2[a, b]\). Then, we can represent f(t) in a series as
$$ f(t) = \frac{a_0}{2} + \sum _{k=1}^\infty a_k\cos \left( \frac{k\pi (t-h)}{H}\right) + b_k \sin \left( \frac{k\pi (t-h)}{H}\right) $$
where
$$ a_k = \frac{1}{H} \int _{a}^b f(t)\cos \left( \frac{k\pi (t-h)}{H}\right) dt , \text{ and } b_k = \frac{1}{H} \int _{a}^b f(t) \sin \left( \frac{k\pi (t-h)}{H} \right) dt. $$
The above constants are given by \(H = (b-a)/2\), \( h = (a+b)/2.\)

Note that Theorem 2.1.2 is just a generalization of Theorem 2.1.1. To see this, let \(a=-\pi \), \(b =\pi \), which implies that \(H = \pi \), and \(h = 0\). Let us add a simplified version of Theorem 2.1.2, where the interval is centered about the origin, or where \(a=-T\) and \(b=T\).

Corollary 2.1.3

Let f(t) be any function in \(L^2[-T, T]\). Then, we can represent f(t) in a series as
$$ f(t) = \frac{a_0}{2} + \sum _{k=1}^\infty a_k\cos \left( \frac{k\pi t}{T}\right) + b_k \sin \left( \frac{k\pi t}{T}\right) $$
Figure 2.1:

Above you see that both the cosine (in green) and the sine (in blue) have exactly one cycle between \(-\pi \) and \(\pi \).

where
$$ a_k = \frac{1}{T} \int _{-T}^T f(t)\cos \left( \frac{k\pi t}{T}\right) dt , \text{ and } b_k = \frac{1}{T} \int _{-T}^T f(t) \sin \left( \frac{k\pi t}{T} \right) dt. $$

Theorem 2.1.1 is relatively easy to remember, while the author cannot remember Theorem 2.1.2 without recreating it. It is much easier to understand than it is to memorize, so let us understand how you can get Theorem 2.1.2 from Theorem 2.1.1 without memorization. To do this, we need pictures.

The key which is illustrated in Figure 2.1 is that the first cosine and the first sine in Theorem 2.1.1 (i.e., the cosine and sine terms with k = 1) have exactly one cycle between \(-\pi \), and \(\pi \). If we make the interval \([a, b] = [-2\pi , 2\pi ]\), then we would have to change the cosine and sine terms to be \(\cos (kt/2 )\) and \(\sin ( kt/2)\). This assures that when t reaches the edge of the interval or \(2\pi \), we will have \( k t/2 = k \pi \). This assures that the first cosine and sine terms of our new series (using \(\cos (kt/2 )\) with \(k=1\)) will also have exactly one cycle.

To understand why this works, imagine a function f(t) expanded on \([-\pi ,\pi ]\). Now, stretch that function to \([-2\pi , 2\pi ]\) by considering f(t / 2). Notice that t/2 travels from \(-\pi \) to \(\pi \) in the time that t goes from \(-2\pi \) to \(2\pi \). All we did by substituting t/2 into the cosine or sine terms is adjust the series that we had formerly expanded, or if
$$ f(t) = \frac{a_0}{2} + \sum _{k=1}^\infty a_k\cos (kt) + b_k \sin (kt) $$
then surely
$$ f(t/2) = \frac{a_0}{2} + \sum _{k=1}^\infty a_k\cos (kt/2) + b_k \sin (kt/2). $$
The one detail is that we do have to calculate the coefficients \(a_k\) and \(b_k\) slightly differently on our new interval. You can, however, do a change of variable in the old formula and get the same thing.

2.1.1 Calculating a Fourier Series

The obvious problem with Theorem 2.1.1 is that we do not just have to calculate one integral, but an infinite number of them. The reality is that oftentimes we can calculate all of these integrals simultaneously. The second reality is that except for a certain number of simple functions, the Fourier Series cannot easily be calculated by hand. The Fourier Series will be evaluated by the computer in general, but that is a topic for a later chapter.

Example 1:

Let us begin with one of the most basic functions for which Fourier Series is used. This is a common function which is oftentimes used to model an on–off switch in electrical engineering. Let us consider what we will call the characteristic function on \([-\pi ,\pi ]\) which we define to be
$$\begin{aligned} \chi (t) = \left\{ \begin{array}{c} 1 \text{ if } |t| < \frac{\pi }{2} \\ 0 \text{ if } |t| > \frac{\pi }{2} \end{array}. \right. \end{aligned}$$
(2.2)
We will use this function often so we will introduce the notation
$$\begin{aligned} \chi _a(t) = \left\{ \begin{array}{c} 1 \text{ if } |t| < a \\ 0 \text{ if } |t| > a \end{array}, \right. \end{aligned}$$
(2.3)
or
$$\begin{aligned} \chi _{[a, b]}(t) = \left\{ \begin{array}{c} 1 \text{ if } |t| \in [a, b] \\ 0 \text{ if } |t| \notin [a, b] \end{array}. \right. \end{aligned}$$
(2.4)
Now, we need to calculate the coefficients
$$ a_k = \frac{1}{\pi } \int _{-\pi }^\pi f(t)\cos (kt) dt , \text{ and } b_k = \frac{1}{\pi } \int _{-\pi }^\pi f(t) \sin ( kt ) dt , $$
from Theorem 2.1.1. There are a couple of things we need to remember from basic calculus to make our lives easier. The first is the definition of even and odd functions.

Definition 2.1.2

(Even and Odd Functions) A function f(t) is said to be even if \(f(t) = f(-t)\). A function f(t) is said to be odd if \(f(t) = -f(-t)\).

The first obvious reason why we care about even and odd functions is that \(\cos (kt)\) is even for all k and \(\sin (kt)\) is odd for all k. Another set of functions which separate nicely into even and odd functions is the monomials \(t^k\). If k is even, \(t^k\) is even. If k is odd, \(t^k\) is odd.

To understand why this helps, remember that if f(t) is odd, \(\int _{-T}^T f(t) dt = 0\). Now, remember that the product of two even functions is even. The product of an even function and an odd function is odd. Finally, the product of an odd function with an odd function is even. These correspond directly to the products of positive and negative numbers, where even is positive and odd is negative, for some rather obvious reasons.

Now, consider the formulas for \(a_k\) and \(b_k\) above. If f(t) is even, then \(f(t) \sin (kt)\) is odd, so the \(b_k\) terms will all be zero. This makes sense because an even function can be represented entirely by cosine (even) terms. Similarly, if f(t) is odd, \(f(t)\cos (kt)\) will be odd, so the \(a_k\) terms are all zero. Thus, the Fourier Series also separates our functions nicely into even and odd terms.

Let us return to calculating the Fourier Series for \(\chi (t)\) as defined above. Notice that \(\chi (t)\) is even, so as a result all of the sine terms will be zero. Let us now consider the \(a_k\) or cosine terms. To begin with this seems difficult, but the key is to separate the integral into the two parts, namely when \(|t| < \frac{\pi }{2}\) and when \(|t| > \frac{\pi }{2}\). Mathematically, this becomes
$$ a_k = \frac{1}{\pi } \int _{-\pi }^\pi \chi (t)\cos (kt) dt = \frac{1}{\pi } \int _{-\pi /2}^{\pi /2} \chi (t)\cos (kt) dt + \frac{1}{\pi } \int _{\pi /2<|t|\le \pi } \chi (t)\cos (kt) dt . $$
Note that \(\chi (t) =1 \) in the first integral and \(\chi (t) = 0\) in the second integral is zero so we have
$$\begin{aligned} a_k= & {} \frac{1}{\pi } \int _{-\pi /2}^{\pi /2} 1\cos (kt) dt + 0 = \frac{1}{\pi } \frac{\sin (kt)}{k} |^{\pi /2}_{-\pi /2}, \end{aligned}$$
(2.5)
$$\begin{aligned}= & {} \frac{1}{\pi } \frac{\sin (k\pi /2) - \sin (-k\pi /2)}{k} = \frac{2 \sin (k\pi /2)}{k \pi }. \end{aligned}$$
(2.6)
Note that we used the odd property of the sine function to finish the last step, namely \(\sin (x)-\sin (-x) = 2 \sin (x)\). This is a fine and complete answer which will allow us to calculate the series without difficulty. Further examination, however, shows that when k is even, the terms are zero, i.e., \(\sin ( m \pi ) = 0\) for all integers m. Examining once again shows that for k odd, \(\sin (k \pi / 2) = (-1)^{(k+1)/2}\). Thus, it is 1 for \(k=1\) and \(-1\) for \(k=3\), and it alternates through the series. This is a nice formula, but notice that (2.5) is not valid for \( k = 0 \), since \(\cos ( 0 t) = 1\), and \(\int 1 dt = t\). This individual case is easily solved, and we get \(\frac{a_0}{2} = \frac{1}{2}\). This is also the average value of \(\chi (t)\) which will always be the case, from the definition of \(\frac{a_0}{2}\). Thus, the series is
$$\begin{aligned} \chi (t) = \frac{1}{2} + \frac{2}{\pi }\cos ( t ) - \frac{2}{3 \pi }\cos (3t) + \frac{2}{5 \pi }\cos ( 5t) - \frac{2}{7\pi }\cos (7t) \dots . \end{aligned}$$
(2.7)
While the expansion formula (2.7) is nice, it is actually harder to program on a computer than the most basic formula which we derived in (2.5). This formula is then
$$ \chi (t) = \frac{1}{2} + \sum _{k=0}^\infty \frac{2 \sin (k\pi /2)}{k \pi }\cos ( kt ) . $$
Note that the above formula has every other term being zero, but it is actually easier to program on a computer. Thus, sometimes trying to reduce the formulas too far actually requires more work later.
We are now going to try to see what this series, or the partial sums of this series look like. No one ever evaluates an infinite sum unless it is a very special sum. By partial sums, we are referring to the approximations
$$\begin{aligned} S_n(\chi (t)) = \frac{1}{2} + \sum _{k=0}^n \frac{2 \sin (k\pi /2)}{k \pi }\cos ( kt ) . \end{aligned}$$
(2.8)
A crucial question in Fourier Analysis is “How quickly do these sums begin to approximate \(\chi (t)\)?” These approximations are shown in Figure 2.2. You can see from this figure that the basic shape of the function is starting to approximate the function. Note that these sums are only guaranteed to approximate the function according to the average error metric \(E_n(\chi ) = \int _{-\pi }^\pi | f(t) - S_n(t) |^2 dt \). A quick examination of the numerics used to generate the plots shows that the errors \(E_n(\chi )\) are 1.57 for \(n = 0\), and 0.29, 0.15, and 0.10 for \(n = 1,3,5\), respectively. Remember that every other term in the approximation is zero, so it does not get better for \(n = 2, 4, 6 \) etc.

To investigate further we check the numerical approximation of \(\chi (t)\) with 20, 50, and 100 terms. The numerical results for the error are 0.03, 0.01, and 0.008. Thus, the error is going to zero, although very slowly. These results are shown in Figure 2.3. We will spend extensive time in the future understanding the rate at which these series converge, and the many implications of these convergence rates.

Example 2:

Before we move on to the next topic, we will address a second example, pointing the way to the solution which is left as an exercise. Let us represent the extremely simple function \(f(t) = t\) in terms of its Fourier Series, on the simple interval from \([-\pi ,\pi ]\). Let us explain why we call this the simple interval. This is because you do not need anything inside \(\cos (kt)\) or \(\sin (kt)\). It can be argued that \([-1,1]\) is a simpler interval, but then you have \(\cos (k\pi t)\), etc. Neither is very difficult.

The coefficients for the function f(t) will be exclusively sine coefficients, since t is an odd function. Recall odd functions only have sine coefficients. If you get confused, just compute them all, and hopefully, the cosine coefficients will be zero. It is good to understand, however, since it saves time and allows one to correct arithmetic mistakes. So our sine coefficients will be
$$ b_k = \frac{1}{\pi } \int _{-\pi }^\pi t \sin (kt) dt . $$
This evaluation is trickier than the prior example. One must remember integration by parts from first-year calculus. This is not all that time consuming and is left as an exercise. We do illustrate the approximations in Figure 2.4.
It is informative that if we consider \(f(t) = t^2\), then we have to do integration by parts twice and three times for \(f(t) = t^3\). This brings up one of the problems with basic Fourier Series. Calculating the integral coefficients by hand
Figure 2.2:

We have plotted the first four nonzero approximations of the Fourier expansion derived above in (2.7). You can see the terms start to approximate the function, although they are not exact with only n = 1, 3, and 5 terms, respectively.

Figure 2.3:

We have plotted the first approximations to \(\chi (t)\) above, with 20, 50, and 100 terms. The terms are starting to approximate the function very closely, but there is still a lot of “ringing” in the approximation which is not present in the function.

Figure 2.4:

We have plotted the first approximations to \(f(t) = t\) on \([-\pi ,\pi ]\) above, with 5 and 50 terms. The terms are starting to approximate the function very closely, but there is still a lot of “ringing” in the approximation which is not present in the function. Note also that the approximations are only valid on \([-\pi ,\pi ]\), and the representation is periodic and not at all valid outside of \([-\pi ,\pi ]\).

is not easy for most functions and impossible for some. As a result, we will study methods to calculate these coefficients by other means, most often with numerical approximations.

2.1.2 Periodicity and Equality

Fourier Series can approximate nearly any function over a finite interval. One should not make the mistake of believing that the Fourier Series will somehow represent the function outside of that predesigned interval. Recall that if the interval is \([-\pi ,\pi ]\), then the Fourier Series takes on the form
$$\begin{aligned} f(t) = \frac{a_0}{2} + \sum _{k=1}^\infty a_k\cos (kt) + b_k \sin (kt). \end{aligned}$$
(2.9)
All of the functions in the above expansion are \(2\pi \) periodic. Specifically, \(\cos (kt) =\cos ( k ( t + m 2\pi ))\) and \(\sin (kt) = \sin (k(t+n2\pi )\). Therefore, we must consider two different things, the function f(t) and the Fourier expansion of f(t), which we will define to be
$$ S(f)(t) = \frac{a_0}{2} + \sum _{k=1}^\infty a_k\cos (kt) + b_k \sin (kt). $$
Since S(f)(t) consists entirely of \(2\pi \) periodic functions, it follows that \(S(f)(t) = S(f)(t+ m 2\pi )\) and is also \(2\pi \) periodic.
Figure 2.5:

We have plotted an approximation to \(\chi (t)\) above, with 50 terms on \([-2\pi , 2\pi ]\). Note that this approximation is relatively good from \([-\pi ,\pi ]\), but does not approximate \(\chi (t)\) outside of the interval. Instead it repeats itself, as we have stated.

For this reason, we can only say that \(f(t) = S(f)\) on the interval \([-\pi ,\pi ]\). This is completely reasonable, because the coefficients \(a_k\) and \(b_k\) only depend on f(t) in that interval. Secondly, remember that f(t) does not need to be periodic at all, and so it is obvious that we cannot expect that S(f)(t) would represent f(t) outside of the defined interval. Perhaps we should refer to S(f)(t) as \(S_\pi (f)(t)\), meaning that it is the Fourier Series on \([-\pi ,\pi ]\), and \(S_{[a, b]}(f)(t)\) to be the Fourier Series on [ab]. When it is important to make the distinction, we will use this altered notation.

To emphasize this point, we should look at \(S_\pi (f)(t)\), or one of its approximations, on \([-2\pi , 2\pi ]\). This is shown in Figure 2.5.

At this time, we need to address the issue of how we are using equality Theorem 2.1.1, and in all of the stated equalities in this chapter. What does it mean that a function
$$ f(t) = \frac{a_0}{2} + \sum _{k=1}^\infty a_k\cos (kt) + b_k \sin (kt)? $$
The right-hand side is an infinite sum, so what we would like to say is that the limit as the number of terms goes to \(\infty \) converges to f(t). Even this is not the correct statement. Let us define
$$ S_n(f) =\frac{a_0}{2} + \sum _{k=1}^n a_k\cos (kt) + b_k \sin (kt).$$
What we really mean is that the error function defined as
$$E_n(f) = \int _{-\pi }^\pi | f(t) - S_n(t) |^2 dt $$
goes to zero as \(n \rightarrow \infty \). Specifically, we know that \( \lim _{n\rightarrow \infty } E_n(f(t)) = 0\) for any function \(f(t) \in L^2[a, b]\). We cannot guarantee that the same is true if we remove the integral in the definition of \(E_n(f)\). Specifically, we will not have \(|f(t) - S_n(t)| \rightarrow 0\) for all \(f(t) \in L^2(a, b)\). Pointwise convergence in \(L^2[a, b]\) does not really make sense however, since we say that two functions are equal if \(\int |f(t) - g(t) |^2 dt = 0\). Remember that integrals do not take into account the value of functions at single points. In general, the partial sums do approach the function f(t) at each specific point, at least with the exception of a set of measure zero, which is beyond the scope of this course. This was proven relatively recently [4].

There are many different books which discuss the relative convergence rates of Fourier Series. These are important and interesting, and the reader is encouraged to look into these topics [1, 3, 17, 20]. They are beyond the scope of this book, however. We will consider only Fourier Series in \(L^2\) of some type or another. We will then use these convergence rates to understand some of the subtleties of the applications.

2.1.3 Problems and Exercises

  1. 1.
    Compute the Fourier Series for the function
    $$\begin{aligned} \chi _{\pi /4}(t) = \left\{ \begin{array}{c} 1 \text{ if } |t| < \frac{\pi }{4} \\ 0 \text{ if } |t| > \frac{\pi }{4} \end{array}, \right. \end{aligned}$$
    (2.10)
    on \([-\pi ,\pi ]\) by using the above example as a guide. Also, plot the first few terms of the expansion (a) on \([-\pi ,\pi ]\) and (b) on \([-2\pi , 2\pi ]\).
     
  2. 2.

    Compute the Fourier Series for \(\chi _{\pi /4}(t)\) on \([-2\pi , 2\pi ]\). Plot this on the interval from \([-6\pi , 6\pi ]\) or a larger interval and compare to the plot on the function expanded on \([-\pi ,\pi ]\). Realize that is should converge on the whole interval \([-2\pi , 2\pi ]\), and you must use the appropriate sine and cosine terms \( \cos (k/2 t) \) and \( \sin (k/2 t)\).

     
  3. 3.

    Compute two Fourier Series for the function \(f(t) = t\) (a) on \([-\pi ,\pi ]\) and (b) on \([-2\pi , 2\pi ]\). Plot the approximations using 5,10, and 15 terms on \([-4\pi , 4\pi ]\).

     
  4. 4.

    (a) Find the necessary cosine and sine terms for expanding on \([-1,1]\) and the appropriate coefficient formulas to approximate a function on \([-1,1]\). (b) Compute the expansion of \(f(t) = t^2\) on \([-1,1]\), and plot the first few terms on \([-3,3]\).

     
  5. 5.

    (a) Figure out the necessary cosine and sine terms and the appropriate coefficient formulas to approximate a function on the interval [1, 3]. (b) Using the result from 4 above, plot the first few terms of the expansion of t on [1, 3] and on \([-1,5]\).

     
  6. 6.

    Find the expansion for \(f(t) = t^3\) on \([-1,1]\). Plot the first few terms on the interval \([-3,3]\).

     
  7. 7.

    Compute the expansion of \(f(t) = \cos (t/2)\) on \([-\pi ,\pi ]\). Plot the first few terms on this interval.

     
  8. 8.

    Compute the expansion of \(f(t) = \cos ^2(t/2)\) on \([-\pi ,\pi ]\). Plot the first few terms on this interval.

     
  9. 9.

    Compute the expansion of \(f(t) = \sin (t/2)\) on \([-\pi ,\pi ]\). Plot the first few terms on this interval.

     
  10. 10.
    Compute the expansion of
    $$\begin{aligned} f(t) = \left\{ \begin{array}{l@{\quad }l} 0 &{} \text{ for } |t| > \pi /2 \\ t+\pi /2 &{} \text{ for } t\in [-\pi /2,0] \\ \pi /2-t &{} \text{ for } t\in [0,\pi /2] \end{array} \right. \end{aligned}$$
    (2.11)
    on the interval \([-\pi ,\pi ]\). Plot the first few terms on this interval.
     
  11. 11.
    Compute the expansion of
    $$\begin{aligned} f(t) = \left\{ \begin{array}{l@{\quad }l} 0 &{} \text{ for } |t| > \pi /2 \\ -1 &{} \text{ for } t\in [-\pi /2,0] \\ 1 &{} \text{ for } t\in [0,\pi /2] \end{array} \right. \end{aligned}$$
    (2.12)
    on the interval \([-\pi ,\pi ]\). Plot the first few terms on this interval.
     
  12. 12.
    Compute the expansion of
    $$\begin{aligned} f(t) = \left\{ \begin{array}{l@{\quad }l} 0 &{} \text{ for } |t| <0 \\ 1 &{} \text{ for } |t| \ge 0 \end{array} \right. \end{aligned}$$
    (2.13)
    on the interval \([-\pi ,\pi ]\). Plot the first few terms on this interval. Plot them on \([-2\pi , 2\pi ]\) also.
     

2.2 Orthogonality and Hilbert Spaces

We simply stated the basics of Fourier Series in the last section. We intentionally presented no proofs, and we also skipped what we will refer to as the geometry of Fourier Series. We will refer to the discussion of orthogonality which we touched on in the first chapter. Orthogonality generally comes up in Linear Algebra. Its extension to studying functions here is straightforward. We will review quickly the basic concepts. The first and most obvious statement or question is “What is the difference between orthogonality and perpendicularity?” The answer is there is no difference. They are interchangeable, but mathematicians and most others generally switch to orthogonality when dealing with higher dimensional vector spaces, i.e., \({\mathbb R}^n\), where \(n> 3\), or function spaces, such as \(L^2[a, b]\) as we defined in the last section.

Let us first begin with some notation. From this point forward, we will cease to recognize the difference between a common dot product, as we have in basic Linear Algebra, and the inner product, which we have yet to define. The dot product of two vectors is given by \(\vec {a} \cdot \vec {b} = \vec {a}^t b = \sum _k a_k b_k \). Namely, we multiply and add. Remember that length (common Euclidean distance) is given by \( | \vec {a} | = \sqrt{ \vec {a}\cdot \vec {a} }\). Recall also that the angle \(\theta \) between two vectors is given by \( \vec {a} \cdot \vec {b} = |a||b|\cos ( \theta ) \).

To define the inner product as we will use it in this book, think first of how we would compare two functions f(t) and g(t). We might start by approximating length by using the dot product, perhaps sampling (or discretizing) the functions to a finite number of points \(t_k\), where \(t_0 = a\) and \(t_n = b\) on an interval [ab]. It follows that the distance between consecutive points would be \(\delta t = (b-a)/n\). Thus, we would like to approximate the length of these functions by their vector counterparts, or say that \(f(t) \cdot g(t) \approx \sum _k f(t_k) g(t_k)\). The one problem is that if we use twice as many points, the length will approximately double. This is recognized by multiplying by the distance between points or using
$$ f(t) \cdot g(t) \approx \sum _k f(t_k) g(t_k) \delta t. $$
We recognize the above from first-year calculus as a Riemann sum. The Fundamental Theorem of Calculus then tells us if we let \(\delta t\) go to zero, the limit should converge (in most cases), and therefore, we have our definition of an inner product

Definition 2.2.1

(Inner Product) If f(t) and g(t) are functions in \(L^2[a, b]\), then we define the inner product between f(t) and g(t) to be
$$ (f(t) \cdot g(t)) \equiv \langle f(t), g(t) \rangle = \int _a^b f(t) \overline{g(t)} dt. $$
Moreover, the length of f(t) is designated to be \( || f(t) || = \sqrt{(} \int _a^b | f(t) |^2 dt = \sqrt{ \langle f, f \rangle }.\)

Note that we had to use the complex conjugate in the inner product definition above. This allows us to deal with complex-valued functions. If we did not use this, note that a function which is purely complex would have a negative distance, which we do not want. We will oftentimes not include the t in the integral and just write \(\langle f , g \rangle \) for the inner product.

Any standard text on linear analysis or Linear Algebra will go into the details of inner products. We encourage the reader to check out these more detailed expositions. The important part here is that \(L^2[a, b]\) will inherit all of the properties that we were used to in Linear Algebra, with the standard dot product. We state the important ones here:
  1. 1.

    (Linearity) If f and g are any two functions in an inner product space, then \(c_1 f + c_2 g\) is also a function in that inner product space.

     
  2. 2.

    If \(f \ne 0\), then \(\Vert f \Vert \ne 0\).

     
  3. 3.

    (Triangle Rule) If f and g are in an inner product space, \( \Vert f + g \Vert \le \Vert f \Vert + \Vert g\Vert \).

     
  4. 4.
    (Cauchy–Schwartz Inequality) If f and g are in an inner product space
    $$\begin{aligned} | \langle f , g \rangle | \le \Vert f \Vert \Vert g \Vert . \end{aligned}$$
    (2.14)
    Moreover, equality holds only if f is a constant multiple of g, or \(f(t) = c g(t)\).
     

Let us recall that the Cauchy–Schwartz inequality is an outgrowth of the standard formula in Linear Algebra \( \vec {a} \cdot \vec {b} = |a||b|\cos (\theta ) \). Since \(\cos (\theta )\) is always less than or equal to one, the inequality holds.

We will now focus on orthogonality, since it is a key which we must keep in mind. Simple functions such as the monomials \( 1, t, t^2, t^3 \dots \) are in \(L^2\) of any interval, but they are not even close to being orthogonal. Let us now formalize this idea which was introduced in Chapter  1.

Definition 2.2.2

Two functions f(t) and g(t) in \(L^2[a, b]\) are said to be orthogonal if \(\langle f, g \rangle = 0\). A collection or set of functions \(\{o_k(t)\}_{k=0}^N\) is said to be orthogonal if for any pair functions from the set \( \langle o_j , o_i \rangle = 0\) as long as \( i \ne j \). In addition, we can say that a set of functions is orthonormal if they are orthogonal, and they all have length one, or \( \Vert o_j \Vert = 1\).

One of the keys to utilizing Fourier Series is that sines and cosines, when adjusted for an interval as in Theorem 2.1.1 and Theorem 2.1.2, are naturally orthogonal. This makes the computation used in Theorem 2.1.1 work. Let us first state the result.

Theorem 2.2.1

(Orthogonality) The functions \(\{\cos (kt)\}_{k=0}^\infty \), and \(\{\sin (kt)\}_{k=1}^\infty \) are orthogonal on the interval \([-\pi ,\pi ]\). In addition, the functions used in Theorem 2.1.2 are also orthogonal on the corresponding interval [ab].

To prove this, we will need to remember the trigonometric addition identities: \(\sin ( \alpha \pm \beta ) =\sin (\alpha )\sin (\beta ) \pm \cos (\alpha )\cos (\beta ) \) and \(\cos ( \alpha \pm \beta ) =\cos (\alpha )\cos (\beta ) \mp \sin (\alpha )\sin (\beta )\).

Proof: We will proceed to prove three things. (1) All of the cosines are orthogonal to all of the sines, and (2) The cosines are orthogonal to each other, and (3) The sines are orthogonal to each other.

Proof of (1): This is easy, since cosine is even and sine is odd, which implies that \(\cos (mt)\sin (nt)\) is odd, so it follows immediately that
$$ \int _{-\pi }^\pi \cos (mt)\sin (nt) = 0 $$
regardless of m or n. This is simple because the integral of an odd function on a symmetric interval is zero, since the right and left halves of the integral are negatives of one another.
Proof of (2): This requires the double angle trigonometric identities. We are considering \(\cos ( mt )\cos (nt )\). If we look at the identities, we have that \(\cos (mt+nt) =\cos (mt)\cos (nt) -\sin (mt)\sin (nt)\) and \(\cos (mt-nt) =\cos (mt)\cos (nt)+\sin (mt)\sin (nt)\). Adding these identities yields
$$\cos (mt+nt)+\cos (mt-nt) =2\cos (mt)\cos (nt).$$
Now substituting, we have
$$\begin{aligned} \int _{-\pi }^\pi \cos (mt)\cos (nt) dt= & {} \int _{-\pi }^\pi \frac{1}{2} (\cos (mt+nt)+\cos (mt-nt) ) dt \\= & {} \frac{1}{2} \left[ \frac{\sin ((m+n)t)}{m+n} + \frac{\sin ( ( m-n) t )}{m-n} \right] _{-\pi }^\pi . \end{aligned}$$
Note that we have assumed in the last integration step that \(m\ne n\). Now, recall that \(\sin (k\pi )= 0\) whenever k is an integer. Thus, both terms on the right are zero. Notice also that if \(m=n\), integration of the \(\cos (mt-nt)=1\) term on the right yields the squared length of \(\cos (mt)\), which is \(\pi \). This yields the \(\frac{1}{\pi }\) term in Theorem 2.1.1.

Proof of (3): This is identical to the Proof of (2) with the exception of using \(\cos (mt+nt)-\cos (mt-nt)\) to cancel the cosine terms on the right.

We have proven orthogonality of the functions in Theorem 2.1.1. The proof for the functions in Theorem 2.1.2 is identical as long as you note the following things. The functions in Theorem 2.1.2 are adjusted so that just in the Proof of part (2) above, the sine terms will be zero at the endpoints. This is due to the fact the sine terms are adjusted to have exactly one cycle for \(k=1\) and multiple cycles \(k>1\) (k is an integer).

2.2.1 Orthogonal Expansions

We have avoided discussions of completeness, or whether or not we have enough functions for the expansions which we are suggesting. An interested student should inquire of this, but it is beyond the scope of this book.

We will prove Theorems 2.1.1 and 2.1.2 given the completeness of those functions. This is an easy process. By completeness, we mean that for any function \(f(t) \in L^2[-\pi ,\pi ]\), we know that there is an expansion of f(t) such that
$$f(t) = \frac{a_0}{2} + \sum _{k=0}^\infty a_k\cos (kt) + b_k\sin (kt) . $$
We will now show that the values for \(a_k\) and \(b_k\) as stated in Theorems 2.1.1 and 2.1.2 are correct. We know that
$$\begin{aligned} \langle f(t),\cos (jt) \rangle= & {} \int _{-\pi }^\pi f(t)\cos (jt) dt \\= & {} \int _{-\pi }^\pi \left( \frac{a_0}{2} + \sum _{k=1}^\infty a_k\cos (kt) + b_k\sin (kt) \right) \cos (jt) dt \\= & {} \int _{-\pi }^\pi \frac{a_0}{2}\cos (jt) dt + \sum _{k=1}^\infty \int _{-\pi }^\pi a_k\cos (kt)\cos (jt)dt + \int _{-\pi }^\pi b_k\sin (kt)\cos (jt) dt. \end{aligned}$$
Note that all of the terms on the right are 0 except the cosine term when \(k=j\), so we have
$$a_k \int _{-\pi }^\pi \cos ^2(kt) dt = \int _{-\pi }^\pi f(t)\cos (kt) dt.$$
Remembering that we proved that right-hand term to be \(\pi \) in the orthogonality section, we have that
$$ a_k = \frac{1}{\pi } \int _{-\pi }^\pi f(t)\cos (kt) dt. $$
Now, we will adjust our notation slightly, so that we can deal with orthonormality. When we deal with the sine and cosine terms \(\sin (kt)\) and \(\cos (kt)\) on \([-\pi ,\pi ]\), their squared lengths in \(L^2[a, b]\) are \(\pi \) as stated above. So to get an orthonormal set of functions, we have to consider \( \frac{1}{\sqrt{\pi }}\sin (kt)\) and \(\frac{1}{\sqrt{\pi }}\cos (kt)\). In addition, the constant term 1 has a length of \(\sqrt{2 \pi }\) so its orthonormal version is \(\frac{1}{\sqrt{2 \pi }}\). Thus, the orthonormal version of the Fourier Series is
$$\begin{aligned} f(t) = \frac{a_0}{\sqrt{2\pi }} + \sum _{k=1}^\infty a_k\frac{\cos (kt)}{\sqrt{\pi }} + b_k \frac{\sin (kt)}{\sqrt{\pi }} , \end{aligned}$$
(2.15)
where
$$ a_k = \int _{-\pi }^\pi \frac{\cos (kt)}{\sqrt{\pi }} f(t) dt , \text{ and } b_k = \int _{-\pi }^\pi \frac{\sin (kt)}{\sqrt{\pi }} f(t) dt , $$
for \(k \ge 0\) and
$$ a_0 = \int _{-\pi }^\pi \frac{1}{\sqrt{2 \pi }} f(t) dt . $$
If we compare the representation in (2.15) to that in (2.1.1), we see that with the exception of the placement of constants, they are identical. The change in notation will be necessary in order to realize the benefits of the orthonormal expansion (2.15).
If we change this to an expansion on \([-T, T]\), then we have
$$\begin{aligned} f(t) = \frac{a_0}{\sqrt{2 T}} + \sum _{k=1}^\infty a_k \frac{\cos (k\pi t/T)}{\sqrt{T}} + b_k \frac{\sin (k\pi t/T)}{\sqrt{T}} , \end{aligned}$$
(2.16)
where
$$ a_k = \int _{-T}^T \frac{\cos (k\pi t/T)}{\sqrt{T}} f(t) dt, \text{ and } b_k = \int _{-T}^T \frac{\sin (k \pi t/T)}{\sqrt{T}} f(t) dt , $$
for \(k \ge 0\) and
$$ a_0 = \int _{-T}^T \frac{1}{\sqrt{2 T}} f(t) dt . $$
More generally, we have the following theorem, whose proof is identical to that above.

Theorem 2.2.2

Let \(\{ o_k \}\) be a complete set of orthonormal functions in \(L^2[a, b]\) then for any function in \(L^2[a, b]\), we have
$$ f(t) = \sum _k \langle f(t), o_k \rangle o_k . $$

This theorem is true in any inner product space, but we state it for our current setting. We will emphasize further generalizations when relevant. From now on, we will refer to any topologically complete inner product space as a Hilbert space. We will not go into the details of topological completeness. It suffices to state that all inner product spaces in this book are Hilbert spaces.

2.2.2 Problems and Exercises:

  1. 1.

    Prove that the representation in (2.15) is correct. Specifically, show that the norm, or length of the functions \(\cos (kt)\) and \(\sin (kt)\) is \(\pi \) on \([-\pi ,\pi ]\), and that the length of 1 is \(2\pi \) on this interval.

     
  2. 2.

    Challenging: Rewrite the expansions in Theorem 2.1.2 so that it is an orthonormal expansion such as in 2.15. Show that these functions are orthogonal on [ab]. Find the norms of these functions on [ab], and give the altered version of Theorem 2.1.2. This theorem should distribute the normalization factors as in (2.15).

     

2.3 The Pythagorean Theorem

One of the oldest and most well-known theorems for elementary students and others dates back to the Greeks. Namely, in a right triangle, or a triangle in which one of the angles is \(90^o\), the squared sum of the length of two sides is the square of the third side, or \(a^2 + b^2 = c^2\). This simple elementary school identity has fundamental importance in Fourier Analysis and Linear Analysis in general. It is also very simple to prove.

Theorem 2.3.1

(Pythagorean Theorem) Let f(t) be a function in a Hilbert space H (such as \(L^2[a, b]\)). Let \(\{o_k\}_k\) be a complete set of orthonormal functions and f(t) be an element of H. We know from Theorem 2.2.2 that
$$ f(t) = \sum _k \langle f(t), o_k \rangle o_k . $$
The Pythagorean theorem states that in addition, the squared length of f, i.e., \(\Vert f \Vert ^2\), can be represented as
$$ \Vert f \Vert ^2 \equiv \langle f , f \rangle = \sum _k | \langle f , o_k \rangle |^2 . $$
In addition, we have that if \( f= \sum _k a_k o_k\) and \(g = \sum _k b_k o_k\), then
$$ \langle f , g \rangle = \sum _k a_k b_k . $$

Stated simply, the squared length of f is identical to the sum of the squared lengths of the sides, or \(\sum _k | \langle f , o_k \rangle |^2\). This is exactly the Pythagorean theorem which is taught at the elementary school. The interesting thing is that the proof can be understood in middle school (perhaps), but it is exactly the same in this highly abstract setting.

Proof: Since we know f is in H, we simply begin with the orthogonal decomposition in Theorem 2.2.2
$$ f(t) = \sum _k \langle f(t), o_k \rangle o_k = \sum _k c_k o_k. $$
where for simplicity of notation we are letting \(c_j = \langle f(t), o_j \rangle \). Now, by the definition of distance or length, we have
$$\begin{aligned} \Vert f \Vert ^2 = \langle f , f \rangle= & {} \left\langle \sum _j a_j o_j , \sum _k a_k o_k \right\rangle \\= & {} \sum _j a_j \left\langle o_j , \sum _k a_k o_k \right\rangle . \end{aligned}$$
In the second line above, we have moved the sum and the constants \(a_j\) outside of the inner product. This is equivalent to moving the sum and constants outside of the integral, which is permitted by the linearity of the integral. We can do this once again with the second sum, and we get
$$\begin{aligned} \Vert f \Vert ^2= & {} \sum _j a_j \left\langle o_j , \sum _k a_k o_k \right\rangle \\= & {} \sum _j \sum _k a_j a_k \langle o_j , o_k \rangle . \end{aligned}$$
Now, recall that the \(\{o_j\}\) are an orthonormal set of vectors. Therefore, \(\langle o_j , o_k \rangle = 0\) if \(j\ne k \), and \( \langle o_k , o_k \rangle = 1\). Thus, the only terms of the double sum which are nonzero are when \(j=k\) so we have a single sum,
$$ \Vert f \Vert ^2 = \sum _k a_k^2 = \sum _k |\langle f(t), o_k \rangle |^2 $$
which is the first portion of what we were going to prove. The inner product identity can be proven in an identical fashion to that above, so we leave that as an exercise.

2.3.1 The Isometry between \(L^2[a, b]\) and \(l^2\)

We will now explore the idea of an isometry. Stated plainly, an isometry is a rule, or transform, which maps one Hilbert space into another Hilbert space, while preserving two things: a) all distance and b) all inner products. A simple example in the two-dimensional plane \(R^2\) is a rotation by \(90^o\), or by any fixed number of degrees. Since the plane remains fixed, with all of the vectors rotating the same number of degrees, it is an isometry. Other examples are flips about the x and y axis, or any other line through the origin.

We will now introduce a couple of definitions which will allow us to express ourselves more easily in the future. We begin the formal definition of a linear operator.

Definition 2.3.1

(Linear Operator) A linear operator, or linear map, from one Hilbert space H to another space G is a rule which uniquely assigns an element \(g \in G\) to each \(h \in H\). In addition, it must be linear. We will generally denote these by \(\mathcal{L} : H \rightarrow G\), meaning that \(\mathcal{L}(f) = g\), with \(f\in H\), and \(g\in G\). Moreover, linearity means that \(\mathcal{L}( c_1 h_1 + c_2 h_2 ) = c_1 \mathcal{L}(h_1) + c_2 \mathcal{L}(h_2)\).

We will now formally define an isometry between Hilbert spaces.

Definition 2.3.2

(Isometry) Let \(\mathcal L\) be a linear operator from one Hilbert space H to another Hilbert space G, i.e., \(\mathcal{L}: H \rightarrow G\). Furthermore, let us denote the inner products in H and G by \(\langle , \rangle _H\) and \(\langle , \rangle _G\). Then, we say that \(\mathcal L \) is an isometry if and only if for any \(f, g \in H\), we have
$$ \langle f , g \rangle _H = \langle \mathcal{L}(f) , \mathcal{L}(g) \rangle _G. $$

The fundamental nature of an isometry is that you can measure distances and angles for functions in one space, say H, by measuring the distances and angles of the images of the functions in the corresponding space, G. This is critical for Fourier Analysis. A great deal of the reason why we use Fourier Series, and in the future the Fourier Transform is that we can often measure something easily in one space, and not so easily in another space. Thus, we choose the place where things are easiest, and then the result extends to the more difficult space. This allows us to analyze the relationships between similar functions, images, or objects in two different ways, and then choose the way which is most clear.

We now define the critical Hilbert space \(l^2\). We are interested in sequences of constants which are either real or complex. We will denote these for now by \(\{ c_k \}\). For the purposes of this book, we will generally have k be either \(k = 0 , 1, 2, 3 \dots \infty \), or \(k = - \infty \dots -2, -1, 0, 1,2, \dots \infty \).

Definition 2.3.3

(Definition of \(l^2\)) Let \(\{c_k\}_k\) be a countable sequence of constants, which may be either real or complex. We say that \(\{ c_k \}_k \in l^2\) if \(\sum _k | c_k |^2 < \infty \). Furthermore, we define the inner product on \(l^2\) to be
$$ \langle \{ a_k \} ,\{ b_k \} \rangle = \sum _k a_k , \overline{b}_k . $$
Note then that the squared distance on \(l^2\) is simply given by \(\sum _k |a_k|^2\).

The notation in the inner product \(\overline{b}\) is the complex conjugate of b. Namely, if b is complex, \( b= \alpha + i \beta \), then \(\overline{b} = \alpha - i \beta \). This is necessary so that the length of complex sequences is positive. Suppose that a sequence has only one nonzero element b, then its squared length would be \(\Vert b\Vert _2^2 = b \overline{b} = ( \alpha + i \beta )( \alpha - i \beta ) = \alpha ^2 + \beta ^2\).

We now will investigate the critical idea of this section. We define by \(\mathcal {F}\) the linear operator which is given by the definition of the Fourier Series. Namely, the orthonormal representation in (2.15) states that
$$\begin{aligned} f(t) = \frac{a_0}{\sqrt{2\pi }} + \sum _{k=1}^\infty a_k \frac{\cos (kt)}{\sqrt{\pi }} + b_k \frac{\sin (kt)}{\pi } , \end{aligned}$$
(2.17)
where
$$ a_k = \int _{-\pi }^\pi \frac{1}{\sqrt{\pi }}\cos (kt) f(t) dt, \text{ and } b_k = \int _{-\pi }^\pi \frac{1}{\sqrt{\pi }}\sin (kt) f(t) dt , $$
for \(k \ge 0\) and
$$ a_0 = \int _{-\pi }^\pi \frac{1}{\sqrt{2 \pi }} f(t) dt . $$
We therefore formally define \(\mathcal{F}\) to be the operator mapping a function \(f(t) \in L^2[a, b]\) to its orthonormal coefficients \(\{a_k\}_{k=0}^\infty \), and \(\{ b_k \}_{k=1}^\infty \). While at first appearance this looks like \(\mathcal {F}\) is mapping f(t) to two sequences in \(l^2\), we do not look at it that way. They are two joined sets of coordinates which we associate with the cosine and sine terms, respectively. We say that \(a_{-k} = b_k\), and then clearly we have exactly one sequence in \(l^2\). We will be switching notation in the future, partially because of the problem between the dual cosine and sine coefficients.

We now state the critical theorem of this section, which is certainly one of the critical ideas of Fourier Analysis.

Theorem 2.3.2

(The Fourier Isometry) The Fourier Series operator \(\mathcal F\) defined above is an isometry between \(L^2[a, b]\) and \(l^2\). Specifically, let f(t) be an element of \(L^2[a, b]\). Then, we have
$$\begin{aligned} \Vert f(t) \Vert _2^2 = \int _a^b |f(t)|^2 dt = \sum _{k=-\infty }^\infty |a_k|^2 ,\end{aligned}$$
(2.18)

where we remember that for \(k<0\), \(a_k = b_{-k}\).

Proof: It turns out that we have already proven this theorem. Note that the Pythagorean Theorem 2.3.1 guarantees us that anytime you have a sequence of complete orthonormal functions in a Hilbert space, you can calculate the length or distance of a function, and inner products between two functions, from the orthogonal coefficients of the function.

Thus, the Pythagorean theorem states that any set of orthonormal functions naturally generates an isometry between the elements of the original Hilbert space, and the Hilbert space \(\{ l^2 \}\).

2.3.2 Complex Notation

We will now switch into complex notation for Fourier Series. We will stay with this notation for most of the rest of this book. The purpose is to eliminate the need to separate the cosine coefficients \(\{a_k\}\) and sine coefficients \(\{ b_k \}\) and the confusion in how the above described Fourier isometry is notated. Recall that if \(i^2 =-1\), then the complex exponential is defined as
$$ e^{\alpha +i\beta } = exp(\alpha + i \beta ) = exp(\alpha ) (\cos (\beta ) + i\sin (\beta ) ) .$$
For the purposes of this book, we will generally be interested in
$$ exp((\alpha +i\beta ) t) = exp(\alpha t) (\cos (\beta t ) + i\sin (\beta t) . $$
First for the purpose of understanding this, let us let \(\alpha = 0\), and take two derivatives of \(exp( i \beta t ) = e^{i \beta t}\) from using the definition of the derivative of the exponential, and the definition of the derivatives of the cosine and sine terms on the right. Note that \( \frac{d^2}{dt^2} exp(i \beta t ) = i^2 \beta ^2 exp(i\beta t ) = (-1) \beta ^2 exp(i\beta t) \) using the definition of the exponential. In addition, you get \(\frac{d^2}{dt^2} exp(i \beta t ) = -1 \beta ^2\cos ( \beta t ) - \beta ^2\sin ( \beta t ) \). Thus, the notation is not artificial, but obeys the rules of differentiation and addition. There are several exercises so if you are new to this you can become familiar.

We will now start using \(\exp ( i kt ) \equiv \cos (kt ) + i\sin (kt )\) as the standard functions on \([-\pi ,\pi ]\). Now, we will state the Fourier Series theorem in complex notation.

Theorem 2.3.3

Let f(t) be any function in \(L^2[-\pi ,\pi ]\). Then, we can represent f(t) in a series as
$$\begin{aligned} f(t) = \frac{1}{\sqrt{2\pi }} \sum _{k=-\infty }^\infty c_k e^{ikt} \end{aligned}$$
(2.19)
where
$$ c_k = \frac{1}{\sqrt{2\pi }} \langle f(t) , e^{ikt} \rangle . $$

Note first of all that this is much simpler than Theorem 2.1.1 (with the exception of getting used to complex notation). There is only one set of coefficients instead of two. There is only one set of functions (rather than cosines and sines). Also note that \(exp(ikt)+exp(-ikt) = 2\cos (kt)\) and \(exp(ikt)-exp(-ikt) = 2\sin (kt)\), so the cosine and sine coefficients and functions are easily recovered from the new functions and coefficients (which will generally be complex).

We must also redefine the inner product for complex-valued functions at this time.

Definition 2.3.4

(Complex Inner Product) The inner product for complex functions of a single variable t on \(L^2[a, b]\) is given by
$$ \langle f(t), g(t) \rangle = \int _a^b f(t) \overline{g}(t) dt . $$
Note that this corresponds with the definition on \(l^2\).

One thing to keep in mind with the complex inner product is that it is linear in the first argument (f(t) above) but conjugate linear in the second argument (g(t) above), i.e., \( \langle f(t) , c g(t) \rangle = \overline{c} \langle f(t), g(t) \rangle \). This is necessary to make distances positive for complex vectors or functions.

We now restate the Fourier isometry from \(L^2[a, b]\) to \(l^2\) using the complex notation, which we will tend to use from now on.

Definition 2.3.5

(Fourier Operator) Given any function \(f(t) \in L^2[a, b]\) and its complex representation (2.19)
$$ f(t) = \frac{1}{\sqrt{2\pi }} \sum _{k=-\infty }^\infty c_k e^{ikt} $$
we define \(\mathcal{F}: L^2[a, b]\rightarrow l^2\) by
$$ \mathcal{F}(f(t)) = \{ c_k \}_k $$
where we have noted that \(\{ c_k \}_k \in l^2\).

Theorem 2.3.4

(Fourier Series Isometry) The Fourier Series operator \(\mathcal F\) defined above is an isometry between \(L^2[a, b]\) and \(l^2\). Specifically, let f(t) and g(t) be elements of \(L^2[-\pi ,\pi ]\). Moreover, let f(t) and g(t) have the representation
$$ f(t) = \frac{1}{\sqrt{2\pi }} \sum _{k=-\infty }^\infty c_k e^{ikt}, $$
and
$$ g(t) = \frac{1}{\sqrt{2\pi }} \sum _{k=-\infty }^\infty d_k e^{ikt}. $$
Then, we have the lengths being equal in both spaces, i.e., \(\Vert f(t) \Vert ^2 = \sum _k |c_k|^2\), \(\Vert g(t) \Vert ^2 = \sum _k |d_k|^2\), and in addition that the inner product
$$ \langle f(t) , g(t) \rangle = \int _a^b f(t) \overline{g}(t) dt = \sum _k c_k \overline{d}_k , $$
can be calculated in either space. Thus, angles and lengths are preserved between the two spaces.

We can state the above theorem for \(L^2[a, b]\), by merely changing the expansions. Thus, this holds on \(L^2[a, b]\).

Proof: The first portion is just a restatement of the Pythagorean theorem. The second portion follows from direct substitution.

Differences between Notations

We know from Theorem (2.3.1) that whenever we have a complete orthonormal set of functions, we have a corresponding orthonormal expansion. The question arises, “Why do we have two for Fourier Series, and how does that change?” Both are the same, and we just want you to recognize the difference.

Specifically, from Theorem 2.1.1, we can write
$$ f(t) = \frac{a_0}{2} + \sum _{k=1}^\infty a_k\cos (kt) + b_k\sin (kt) $$
where
$$ a_k = \frac{1}{\pi } \int _{-\pi }^\pi f(t)\cos (kt) dt , \text{ and } b_k = \frac{1}{\pi } \int _{-\pi }^\pi f(t)\sin ( kt ) dt . $$
But we later introduced the representation in (2.15)
$$ f(t) = \frac{a_0}{\sqrt{2\pi }} + \frac{1}{\sqrt{\pi }} \sum _{k=1}^\infty a_k\cos (kt) + b_k\sin (kt) , $$
where
$$ a_k = \int _{-\pi }^\pi \frac{1}{\sqrt{\pi }}\cos (kt) f(t) dt , \text{ and } b_k = \int _{-\pi }^\pi \frac{1}{\sqrt{\pi }}\sin (kt) f(t) dt ,$$
for \(k\ge 1\) and \(a_0 = 1/\sqrt{2\pi } \int f(t) dt\). They are the same. Note that in the second representation, the \(\sqrt{\pi }\) is split, and in the first, it is combined. Why should we have two notations? The representation in (2.15) is necessary to make sure the isometry is preserved. The original representation in Theorem 2.1.1 is somewhat cleaner. Moreover, you will eventually see different representations and you need to know how they relate. They are all the same, but the notation is changed.
To make this clear, let us consider the following project. Given a complete and orthogonal set of functions \(\{f_k \}\) in a Hilbert space \(\mathcal{H}\), we want to have orthonormal expansions. This means we must normalize the functions. We do this very simply by defining
$$ o_k = \frac{f_k}{\Vert f_k\Vert } .$$
Now, for any function \(f \in \mathcal{H}\), we have
$$\begin{aligned} f= & {} \sum _k \langle f, o_k \rangle o_k \nonumber \\= & {} \sum _k \langle f, \frac{f_k}{\Vert f_k\Vert } \rangle \frac{f_k}{\Vert f_k\Vert } \end{aligned}$$
(2.20)
$$\begin{aligned}= & {} \frac{1}{\Vert f_k \Vert ^2 } \sum _k \langle f, f_k \rangle f_k . \end{aligned}$$
(2.21)
The difference in representations between (2.21) and (2.20) is exactly the difference between the Fourier references above. To obtain the isometry, we have to use (2.20) and the coefficients of (2.20).

2.3.3 Estimating Truncation Errors

Only under rare circumstances are we able to turn infinite series into a concrete equation. Thus, we are often forced to estimate an infinite Fourier Series with its finite approximation. Let us refer to the Fourier summation as S(f), and the truncated summation \(S_N(f)\), and the error in truncation as \(E_N(f)\), so we have
$$\begin{aligned} f(t)= & {} S(f) = \frac{a_0}{2} + \sum _{k=1}^\infty a_k\cos (kt) + b_k\sin (kt) \nonumber \\= & {} \left( \frac{a_0}{2} + \sum _{k=1}^N a_k\cos (kt) + b_k\sin (kt) \right) + \left( \sum _{k=N+1}^\infty a_k\cos (kt) + b_k\sin (kt) \right) \nonumber \\= & {} S_N(f) + E_N(f). \end{aligned}$$
(2.22)
Similarly if we are using complex notation, let us define
$$\begin{aligned} f(t)= & {} S(f) = \frac{1}{\sqrt{2\pi }} \sum _{k=-\infty }^\infty c_k e^{ikt} \nonumber \\= & {} \left( \frac{1}{\sqrt{2\pi }} \sum _{|k|\le N} c_k e^{ikt} \right) + \left( \frac{1}{\sqrt{2\pi }} \sum _{|k|>N} c_k e^{ikt} \right) \nonumber \\= & {} S_N(f) + E_N(f). \end{aligned}$$
(2.23)
We are primarily interested in estimating \(f(t) - S_N(f) = E_N(f)\). Generally, the easiest way to estimate this is in the metric,
$$\begin{aligned} \Vert f(t) - S_N(f) \Vert _2^2 = \int _{-\pi }^\pi |f(t)-S_N(f)|^2 dt = \int _{-\pi }^\pi |E_N(f)|^2 dt . \end{aligned}$$
(2.24)
Thus, if we want to estimate our error, we can estimate either of the two quantities at the right above. By the Fourier isometry, each of those quantities can be measured at least two ways.
Estimating \(\int _{-\pi }^\pi |E_N(f)|^2 dt\) by calculating the integral in time requires the numerical estimation integral, which creates some difficulties, but is possible. Estimating the Fourier Transform via the isometry is sometimes much easier. In addition, creative use of the isometry will allow us to measure this in a number of ways. One common way is to realize that
$$ \Vert E_N(f) \Vert ^2 = \sum _{|k|> N} |c_k|^2 . $$
Since this is an infinite integral, it is not generally easy to calculate in closed form. However, we know that
$$ \Vert f \Vert ^2 - \sum _{|k|\le N} |c_k|^2 = \sum _{k>N} |c_k|^2. $$
Thus, we can calculate the norm of f by integrating in time directly, and then subtract the finite sum of coefficients until the values on the right are sufficiently small. This is one of the advantages of the isometry. You can measure things in different ways, and some are much more efficient than others.

An Example:

As an example, we will return to the first Fourier Series we calculated. Recall that
$$ \chi _{\pi }(t) = \frac{1}{2} + \sum _{k=0}^\infty \frac{2 \sin (k\pi /2)}{k \pi }\cos ( kt ) , $$
on \([-\pi ,\pi ]\) where
$$\begin{aligned} \chi _{\pi }(t) = \left\{ \begin{array}{c} 1 \text{ if } |t| < \pi \\ 0 \text{ if } |t| > \pi \end{array}, \right. \end{aligned}$$
(2.25)
Now, we would like to calculate how many coefficients it will take to have the error \(\Vert E_N(f) \Vert < .01\). We must first switch to the orthonormal representation
$$ \chi _{\pi }(t) = \frac{\sqrt{2\pi }}{2} \frac{1}{\sqrt{2\pi }} + \sum _{k=0}^\infty \left( \frac{2 \sin (k\pi /2)}{k \sqrt{\pi }}\right) \frac{\cos ( kt )}{\sqrt{\pi }} , $$
where we have normalized the functions 1 and \(\cos (kt)\) and appropriately adjusted the coefficients. Now, by the isometry, we know that \(\Vert \chi (t) \Vert ^2\) is equal to
Figure 2.6:

We plot the convergence of the series (2.26) above. The error is represented by the difference of the squared integral of the function, at top, and the asymptotically approaching sum of squared coefficients, below. While these two do asymptotically approach each other, this convergence is very slow.

the sum of the squared coefficients, or
$$\begin{aligned} \int _{-\pi }^\pi | \chi _{\pi }(t) |^2 dt = \pi = \frac{\pi }{2} + \sum _{k=1}^\infty \left( \frac{2 \sin (k\pi /2)}{k \sqrt{\pi }}\right) ^2 . \end{aligned}$$
(2.26)
This leads us to the equality
$$ \sum _{k=1}^\infty \left( \frac{2 \sin (k\pi /2)}{k \sqrt{\pi }}\right) ^2 =\sum _{k=1}^\infty \frac{4 \sin ^2(k\pi /2)}{k^2 \pi } = \frac{\pi }{2}. $$
The question is, how fast does this happen? By our discussions above
$$\begin{aligned} \Vert E_N(f) \Vert ^2 = \pi - \left( \frac{\pi }{2} + \sum _{k=1}^N \frac{4 \sin ^2(k\pi /2)}{k^2 \pi }\right) . \end{aligned}$$
(2.27)
This can be quickly determined numerically. In Figure 2.6, we see this above sum converging to \(\pi \). A quick check shows that the absolute squared error \(\Vert E_N(f) \Vert ^2\) is less than .01 after 64 terms. Recall, however, that \(\Vert E_N(f) \Vert ^2\) is much smaller than \(\Vert E_N(f) \Vert \). To check \(\Vert E_N(f) \Vert \), we must take the square root of (2.27), and we find that we need 6368 terms. Thus, this sum converges very slowly, if you want a lot of accuracy. This error level would be very hard to check with a direct numerical integration.

2.3.4 Problems and Exercises:

  1. 1.

    Prove that if f(t) and \(g(t)\in L^2[a, b]\), then for any constants \(h(t) = \alpha f(t) + \beta g(t) < \infty \).

     
  2. 2.

    Use basic trigonometric identities to verify that \(e^{ix} e^{iy} = e^{i(x+y)}\).

     
  3. 3.

    Prove the inner product statement in Theorem 2.3.1.

     
  4. 4.
    Show that the representation in Theorem 2.3.3 is a valid orthonormal representation. Specifically, show that the functions
    $$\frac{1}{\sqrt{2\pi }} e^{ikt}$$
    are orthonormal on \([-2\pi , 2\pi ]\).
     
  5. 5.

    Find the orthonormal representation for Theorem 2.1.2, such as we rewrote Theorem 2.1.1 in Theorem 2.3.3. First, find the appropriate complex exponentials for [ab], as are suggested by the sine and cosines of Theorem 2.1.2. Make sure that your altered complex exponentials are orthogonal. Add in the normalization factors which will make them orthonormal. Finally, write the final form of the new theorem, as in Theorem 2.3.3.

     
  6. 6.

    Verify that the Fourier isometry holds on \([-\pi ,\pi ]\) for \(f(t) = t\). To do this, (a) calculate the coefficients of the orthogonal Fourier Series from representation in (2.17), (b) calculate the sum of the squared coefficients, and (c) calculate the norm of the function as \(\int _{-\pi }^\pi |f(t)|^2 dt.\) How many terms in the Fourier Series are necessary to have the isometry be under \(5 \%\)? How many until you are under \(3 \%\), or \(1 \%\)?

     
  7. 7.

    Verify that the Fourier isometry holds on \([-\pi ,\pi ]\) for \(f(t) = \chi _{\pi /4}(t)\). To do this, (a) calculate the coefficients of the orthogonal Fourier Series from representation in (2.17), (b) calculate the sum of the squared coefficients numerically, or analytically if possible, and (c) calculate the norm of the function as \(\int _{-\pi }^\pi |f(t)|^2 dt.\) How many terms in the Fourier Series are necessary to have the isometry be under \(5 \%\)? How many until you are under \(3 \%\), or \(1 \%\)?

     
  8. 8.

    Verify that the Fourier isometry holds on \([-\pi ,\pi ]\) for \(f(t) = t^2\). To do this, (a) calculate the coefficients of the orthogonal Fourier Series from representation in (2.17), (b) calculate the sum of the squared coefficients numerically, or analytically if possible, and (c) calculate the norm of the function as \(\int _{-\pi }^\pi |f(t)|^2 dt.\) How many terms in the Fourier Series are necessary to have the isometry be under \(5 \%\)? How many until you are under \(3 \%\), or \(1 \%\)?

     

2.4 Differentiation and Convergence Rates

We now get to a basic idea of Fourier Analysis. It turns out that the Fourier isometry makes this an easy exercise. Let us suppose that we have
$$ f(t) = \sum _{k=-\infty }^\infty c_k e^{ikt}, $$
as above. The question comes up, can we differentiate f(t) in a term by term manner, namely is
$$\begin{aligned} f'(t) = \sum _{k=-\infty }^\infty (ik) c_k e^{ikt}? \end{aligned}$$
(2.28)
For the purpose of this book, we are only interested in these functions as functions of the space \(L^2[-\pi ,\pi ]\). A simple answer to this question is that if the sum on the right only has a finite number of terms, then \(f'\) can be represented as the above series always. The linearity of the derivative, or the fact that \(\frac{d}{dt} ( a f(t) + b g(t) ) = a f'(t) + bg'(t) \), guarantees this. This becomes trickier when the sum has an infinite number of terms, which is almost always the case.
Let us look at this question as a question about the isometry. Namely, instead of considering f(t) and whether or not it has derivatives in \(L^2[a, b]\) let us consider the question, “Do the Fourier coefficients corresponding to \(f'(t)\) exist in \(l^2\)?” or is
$$\begin{aligned} \Vert f'(t) \Vert ^2 = \sum _{k=-\infty }^\infty |(ik)^2 c_k |^2 = \sum _{k=-\infty }^\infty k^2 |c_k |^2 < \infty . \end{aligned}$$
(2.29)
If the sum at the right is finite, then there certainly is a function, which we will call h in \(L^2[-\pi ,\pi ]\) which has that Fourier expansion. The remaining question is whether that function is \(f'(t)\)?

2.4.1 A Quandary between Calculus and Fourier Analysis

Let us remember integration by parts, where \(\int f'g + f g' dt = fg \) or \( \int f'g = fg - \int fg' dt\). Remember the definition of the complex inner product, and the coefficients \(c_k\), which gives us the coefficients \(d_k\) for \(f'(t)\) as
$$\begin{aligned} d_k= & {} \langle f'(t), \overline{e^{ikt}} \rangle = \int f'(t) e^{-ikt} dt \\= & {} f(t) e^{i kt} |_{-\pi }^\pi + \int f(t) (ik) e^{-ikt} dt \\= & {} f(t) e^{ikt}|_{-\pi }^\pi + (ik) c_k \\= & {} ( -f(\pi )+f(-\pi ) ) + (ik) c_k. \end{aligned}$$
Thus, the coefficients of h, which is the derivative of the Fourier Series of f, would differ from those of \(f'\) if \(f(\pi ) \ne f(-\pi )\). This is where the quandary of Fourier Analysis and \(L^2[-\pi ,\pi ]\) becomes a little tricky. Let us define \(f_1(t) = f(t)\) everywhere except at \(\pi \) and \(-\pi \), where \(f_1(t)=0\) for the sake of argument. Then, the Fourier coefficients for \(f_1(t)\) would be identical to those of h. But f(t) and \(f_1(t)\) are identical except at two points, thus
$$ \Vert f(t) - f_1(t) \Vert ^2 = \int |f(t) - f_1(t)|^2 dt = 0 $$
so they are the same function in \(L^2[a, b]\). Thus, one could argue that the coefficients of h are the coefficients of f.

This is not a valid argument for functions in \(L^2[-\pi ,\pi ]\). If we were dealing with only the continuous functions, then the argument would be valid. The problem is that the integration by parts above assumes that \(f'(t)\in L^2[-\pi ,\pi ]\), but nothing more about \(f'(t)\). We utilized the Fundamental Theorem of Calculus, but that assumes that \(f'(t)\) is continuous. A very extensive resolution of this quandary is given in Tolstov’s book [20]. The theorems of Jackson and other approaches are elegantly described in Cheney [5]. The above argument is “morally” correct, and the details have been extensively studied. The reader is encouraged to look more extensively into these details. For the purposes of this book, we will concentrate on the criterion of equation (2.29). This is sufficient to understand Fourier Series, derivatives, and the rates of decay of Fourier Series, which are essential for the applications in later chapters.

2.4.2 Derivatives and Rates of Decay

Let us return to a basic exponential series
$$f(t) = \sum _{k=-\infty }^\infty c_k e^{ikt}$$
and differentiate it in a term by term manner to obtain the hopeful representation (2.28)
$$f'(t) = \sum _{k=-\infty }^\infty (ik) c_k e^{ikt}.$$
The function \(f'(t)\) as represented in (2.28) can only exist in \(L^2[-\pi ,\pi ]\) if
$$ \Vert f'(t) \Vert ^2 = \sum _{k=-\infty }^\infty |(ik)^2 c_k |^2 = \sum _{k=-\infty }^\infty k^2 |c_k |^2 < \infty . $$
Let us consider the consequences of this equation. Recall the key factors for a sequence to converge. Most notably consider the sum of the series \(\sum _k \frac{1}{k} = \infty \). This can be proven by any number of methods. More generally, we can say that
$$ \sum _{k=1}^\infty \frac{1}{k^{1+\epsilon }} $$
is finite whenever \(\epsilon > 0\) and is infinite whenever \(\epsilon \le 0\). Thus, the dividing line between convergence and divergence is \(\frac{1}{k}\).

Now, let us consider some notation. We will use this to consider more general sequences, or to compare one sequence to another. This will be necessary to understand the nature of Fourier coefficients.

Definition 2.4.1

Given a reference sequence \(\{b_k\}\), we say that another sequence \(\{ a_k \}\) is \(O( b_k )\) if for some finite M the inequality
$$ \frac{ a_k }{b_k}< M < \infty ,$$
holds for all k. The above definition is oftentimes replaced by the limit superior (which we will not go into) and is equivalent to the above. Thus, an equivalent definition is
$$ \overline{\lim }_k \frac{a_k}{b_k}< M < \infty . $$
Moreover, we say that \(\{a_k\}\) is \(o( b_k ) \) if
$$ \lim _{k\rightarrow \infty } \frac{a_k}{b_k} = 0 . $$

These are terms as little “o” and big “O” notation. Thus, if one sequence is smaller than another in the limit, the little “o” applies implying one is smaller than the other. If the sequences are comparable, then the “O” notation applied, implying that they are of similar magnitude, up to a constant M of some size (which might be huge). Note that \(O(b_k)\) is weaker than \(o(b_k)\), so anytime something is little “o” to another sequence it is also big “O”, with the constant M being zero. The little “o” notation is therefore stronger and preferable when available.

Returning to our equation (2.29), we have that
$$\begin{aligned} \sum _{k=-\infty }^\infty k^2 |c_k |^2 < \infty . \end{aligned}$$
(2.30)
if the derivative representation \(f'(t)\) exists in \(L^2[a, b]\). This implies that \(k^2 |c_k|^2 = o\)(1/k), or that series \(\{ k^2 |c_k|^2 \}\) is asymptotically smaller than the series 1/k since the sum of the terms 1/k is infinite. If \(k^2 |c_k|^2\) were not o(1/k), then the sum of these coefficients would have to be infinite (this is false). Thus, it follows that \(|c_k|^2 = o({1}{/}{k^3})\) or that
$$\begin{aligned} c_k = o\left( \frac{1}{k^{3/2}}\right) . \end{aligned}$$
(2.31)
The previous argument is morally correct, but not true. To see this let us define a sequence \(c_k\) to be 0 whenever \(k \ne n^2\) for some n where n is an integer. Let \(c_k = 1/k = 1/n^2\) whenever \(k = n^2\) with n an integer. Then, the condition in (2.30) would be valid, since the terms would be \(\sum _n 1/n^2 < \infty \).

There are many different varieties of the statements for defining when a function is differentiable in terms of its coefficients. Equation (2.30) is correct. Trying to move it to limit equations on the \(c_k\)s becomes difficult mathematically and has led to an incredible number of true theorems. They are all within a very small \(\epsilon \) of the general rule, which is illustrated in (2.31). They are all different, however, and cause confusion. Equation (2.30) is a very good and true guideline. One is referred to [5, 20] and many other books and publications for the details of more exact relationships.

Higher Derivatives

Let us now consider the higher derivatives. Returning to our basic complex Fourier Series
$$ f(t) = \sum _{k=-\infty }^\infty c_k e^{ikt}, $$
we can differentiate this n times in a term by term manner to obtain a series for
$$f^{(n)}(t) = \sum _{k=-\infty }^\infty (ik)^n c_k e^{ikt}. $$
This function does not exist in \(L^2[-\pi ,\pi ]\), however, unless
$$ \sum _{k=-\infty }^\infty k^{2n} |c_k|^2 < \infty . $$
This statement is correct and flows both ways. We have a theorem, specifically

Theorem 2.4.1

Let \(f(t) \in L^2[-\pi ,\pi ]\) have the Fourier representation
$$ f(t) = \sum _{k=-\infty }^\infty c_k e^{ikt}. $$
Then, the \(n^{th}\) derivative of f(t), \(f^{(n)}(t)\) exists in \(L^2[-\pi ,\pi ]\) if and only if
$$\begin{aligned} \sum _{k=-\infty }^\infty k^{2n} |c_k|^2 < \infty \end{aligned}$$
(2.32)
and then it has the representation
$$f^{(n)}(t) = \sum _{k=-\infty }^\infty (ik)^n c_k e^{ikt}. $$

Let us investigate the consequences of the series in (2.32). As before, a good rule of thumb is that the series \(k^{2n} |c_k|^2\) should be asymptotically smaller than the series 1/k, since the sum of 1/k is infinite. This would imply that \(k^{2n} |c_k|^2 = o\)(1/k), or that \(|c_k|^2 = o(1/k^{2n+1})\), implying that \(|c_k| = o(1/k^{n+1/2})\). While this is essentially correct, one can make up counterexamples which as we did in before showing that we can have convergence without this condition.

We can state something more definitive, without being exhaustive on all of the conditions necessary to have a term by term derivative.

Theorem 2.4.2

Let \(f(t) \in L^2[-\pi ,\pi ]\) have the Fourier representation
$$ f(t) = \sum _{k=-\infty }^\infty c_k e^{ikt}. $$
Then, the \(n^{th}\) derivative of f(t), \(f^{(n)}(t)\) exists in \(L^2[-\pi ,\pi ]\) if
$$\begin{aligned} |c_k|^2 = o(1/k^{2n+1+\epsilon }) \end{aligned}$$
(2.33)
for any \(\epsilon >0,\) and it will have the valid Fourier representation
$$f^{(n)}(t) = \sum _{k=-\infty }^\infty (ik)^n c_k e^{ikt}. $$

This clears up some of the problems. We do not claim to be exhaustive on all of the possible conditions which may make term by term differentiation work. Once again, please refer to the references at the end of this book, and the many other possible Fourier Analysis resources. We believe that this basic understanding is enough to move forward.

2.4.3 Fourier Derivatives and Induced Discontinuities

We will now investigate the decay rates via some examples. We first recall that via our discussion in Sect. 2.4.1, a function \(f(t) \in L^2[-\pi ,\pi ]\) will not have a valid term by term Fourier representation of its derivative unless \(f(\pi ) = f(-\pi )\). We would also like to have the Fundamental Theorem of Calculus hold, or we would like
$$ f(t) = f(-\pi ) + \int _{-\pi }^t f'(x) dx . $$
Using this as our definition of a Fourier derivative, we will now analyze several examples to see if term by term differentiation is consistent with this definition, and the decay rates are appropriate.

First Example: Let us consider the simple example of \(f(t) = t\) on \([-\pi ,\pi ]\). While this function seems to be absolutely continuous, its periodic extension, illustrated in Figure 2.4, is discontinuous at every odd multiple of \(\pi \). It fails the basic test \(f(\pi ) = f(-\pi ) ?\), and thus the decay rate cannot be fast. We will leave the prediction of this decay rate and the verification as an exercise.

Second Example: We will now consider the second simple example of \(f(t) = t^2\) on \([-\pi ,\pi ]\). This function does satisfy \(f(\pi ) = f(-\pi )\) and therefore
Figure 2.7:

We have plotted the periodic extension of \(f(t) = t^2\) from \([-\pi ,\pi ]\) to \([-3\pi , 3\pi ]\). Note that the periodic extension is continuous.

since it is continuous, and its derivative is \(f'(t) = 2t\), we expect its series to converge much faster. Certainly, this satisfies the Fundamental Theorem of Calculus. Once again, we will let the reader predict the decay rate, and verify this with the Fourier expansion.

2.4.4 Problems and Exercises:

  1. 1.

    (a) Predict the decay rate of the coefficients for the Fourier expansion of \(f(t) = t\) on \([-\pi ,\pi ]\). (b) Calculate the Fourier Series for this and compare this to your prediction. (c) Were you correct?

     
  2. 2.

    (a) Predict the decay rate of the coefficients for the Fourier expansion of \(f(t) = t^2\) on \([-\pi ,\pi ]\). (b) Calculate the Fourier Series for this and compare this to your prediction. (c) Were you correct?

     
  3. 3.
    Consider the function
    $$\begin{aligned} f(t) = \left\{ \begin{array}{l@{\quad }l} t+1 &{} \text{ for } t \in [-1,0] \\ 1 -t &{} \text{ for } t\in [0,1] \end{array} \right. \end{aligned}$$
    (2.34)
    (a) Does f(t) have a valid Fourier derivative in \([-1,1]\)? (b) What is that derivative if it exists? (c) Calculate the Fourier Transform of the function f(t) on \([-1,1]\). (d) Take the term by term Fourier Transform of this function. (e) Plot this function and state whether it is consistent with the derivative of the function.
     
  4. 4.
    Challenging: Consider the function \(g(t) = f(t)^2\) where
    $$\begin{aligned} f(t) = \left\{ \begin{array}{l@{\quad }l} t+1 &{} \text{ for } t \in [-1,0] \\ 1 -t &{} \text{ for } t\in [0,1] \end{array} \right. \end{aligned}$$
    (2.35)
    (a) Does g(t) have a valid Fourier derivative in \([-1,1]\)? (b) What is that derivative if it exists? (c) Calculate the Fourier Transform of the function g(t) on \([-1,1]\). (d) Take the term by term Fourier Transform of this function. (e) Plot this function and state whether it is consistent with the derivative of the function.
     
  5. 5.
    Consider the function
    $$\begin{aligned} f(t) = \left\{ \begin{array}{l@{\quad }l} 0&{} \text{ for } |t| \ge \pi /2 \\ \cos (t) &{} \text{ for } |t| \le \pi /2 \end{array} \right. \end{aligned}$$
    (2.36)
    (a) Does f(t) have a valid Fourier derivative in \([-\pi ,\pi ]\)? (b) What is that derivative if it exists? (c) Calculate the Fourier Transform of the function f(t) on \([-\pi ,\pi ]\). (d) Take the term by term Fourier Transform of this function. (e) Plot this function and state whether it is consistent with the derivative of the function.
     

2.5 Sine and Cosine Series

We have talked about Fourier Series on \(L^2[a, b]\). By that we are implicitly using both sines and cosines. We have talked about adjusting the sines and cosines to the length of the interval. In addition, we have also talked about the fact that an even function, when expanded about 0, will have only cosine terms, and an odd function, expanded about 0, will have only sine terms.

We will now look to expand functions which are neither odd, nor even, in either cosine or sine series. This cannot be done in the manner described until now in this chapter.

To do this, let us return to an example we had earlier, namely \(f(t) = t\). We expanded this on \([-\pi ,\pi ]\) using the Fourier Series in Figure 2.4. Since this function was odd, this expansion involved only sine terms. Now, we would like to consider expanding it on the half interval \([0,\pi ]\). The earlier sine expansion will obviously still converge on this interval.

We can also expand this function on \([0,\pi ]\) using cosine terms, should we choose. We do this by considering the function \(g(t) = |t|\) on \([-\pi ,\pi ]\) and expanding it in a traditional Fourier Series. Because g(t) is even, this expansion will have only cosine terms. But since \(g(t) = f(t)\) on \([0,\pi ]\), this cosine expansion will converge to f(t) on \([0,\pi ]\).

Thus, we have a way to expand a function f(t) on \([0,\pi ]\) using either a cosine or sine series.

  • If you want to expand in a cosine series, you define
    $$\begin{aligned} g_e(t) = \left\{ \begin{array}{c} f(t) \text{ if } |t| > 0 \\ f(-t) \text{ if } |t| < 0 \end{array}, \right. \end{aligned}$$
    (2.37)
    Thus, you are artificially creating a g(t) which is an even extension of f(t).
  • If you want to expand in a sine series, you define
    $$\begin{aligned} g_o(t) = \left\{ \begin{array}{c} f(t) \text{ if } |t| > 0 \\ -f(t) \text{ if } |t| < 0 \end{array}, \right. \end{aligned}$$
    (2.38)
    Thus, you are artificially creating a g(t) which is an odd extension of f(t).

The Fourier Series for both of these functions will converge on \([-\pi ,\pi ]\), and thus will both be equal to f(t) on \([0,\pi ]\) since \(g_e(t) = g_o(t) = f(t)\) for \(t \in [0,\pi ]\).

The Fourier coefficients will merely be twice the coefficients they would be in the original series, or we have a cosine series which is
$$ f(t) = \frac{a_0}{2} + \sum _{k=1}^\infty a_k \cos (kt)$$
and
$$ f(t) = \sum _{k=1}^\infty b_k \sin (kt), $$
where
$$ a_k = \frac{2}{\pi } \int _{0}^\pi f(t)\cos (kt) dt , \text{ and } b_k = \frac{2}{\pi } \int _{0}^\pi f(t) \sin ( kt ) ) dt. $$
Example
We will now illustrate simply expanding \(f(t) = t\) in a cosine series on \([0,\pi ]\). This means that we are implicitly expanding \(g(t) = |t|\) on \([-\pi ,\pi ]\). Our coefficients are calculated simply from the above formula using integration by parts. Thus when k = 0
$$ a_0 = \frac{2}{\pi } \int _0^\pi t dt = \frac{2}{\pi } t^2/2 |_0^\pi = \pi , $$
so the average value is \(a_0/2 = \pi /2\) as expected.
$$\begin{aligned} a_k= & {} \frac{2}{\pi } \int _0^\pi t \cos (kt) dt \nonumber \\= & {} \frac{2}{\pi }\left( t \frac{ \sin (kt) }{k}\left. \right| _0^\pi - \int _0^\pi 1 \frac{ \sin (kt) }{k} dt \right) \nonumber \\= & {} \frac{2}{\pi }\left( 0 - \frac{ -\cos (kt)}{k^2}\left. \right| _0^\pi \right) \nonumber \\= & {} \frac{2}{k^2 \pi } ( \cos (k\pi )-\cos (0) ) = \frac{2}{k^2 \pi } ( \cos (k\pi )-1). \end{aligned}$$
(2.39)
Note that the terms at the right will be zero for k even. Thus, we have the cosine series
$$\begin{aligned} t = \frac{\pi }{2} + \sum _{k=1}^\infty \frac{2}{k^2 \pi } ( \cos (k\pi )-1) \cos (kt) , \end{aligned}$$
(2.40)
which is valid on \([0,\pi ]\).
Now, let us examine the partial sum approximations which are shown in Figure 2.8. These are remarkably good, for very few terms. Recalling the approximations to \(f(t) = t\) with the sine approximations on \([-\pi ,\pi ]\), these converge much more quickly. The question is why? The answer is simple. With the sine approximation on \([-\pi ,\pi ]\), there is an induced jump discontinuity at the endpoint. By periodizing t, you are eliminating the discontinuity, and trading
Figure 2.8:

We display approximations to \(f(t) = t\) with a cosine series and sine series on \([0,\pi ]\). Note that with just only 10 terms, the cosine approximation is very good. With sine series is not good with 50 or more terms.

it for only a discontinuity in the derivative. Thus, we are now dealing with a continuous and not a discontinuous function. This makes the approximations much better.

Let us remember at this point that we are approximating an even extension of t on \([0,\pi ]\), or the two \(\pi \) periodic extension of t. The odd extension of t would have simply been t, but which is \(2 \pi \) periodic. Thus, let us examine the two periodic extensions, \(g_e(t)\) and \(g_o(t)\) in Figure 2.9.

Let us go back and examine the cosine representation for t shown in (2.40). An obvious reason that it converges quickly is that the coefficients are \(2/k^2\). The sine representation for t (exercise 3 in Section 2.1.3) has coefficients that only decay like 1/k. Thus, it is natural that this series converges faster. We have talked about series converging at a faster rate if and only if they have derivatives. Let us examine the term by term derivative of equation (2.40) or
$$\begin{aligned}&\frac{d}{dt} \left( \frac{\pi }{2}+ \frac{2}{k^2 \pi } \sum _{k=1}^\infty ( \cos (k\pi )-1) \cos (kt) \right) \nonumber \\= & {} \frac{2}{k^2 \pi } \sum _{k=1}^\infty ( \cos (k\pi )-1) \left( -k\sin (kt) \right) \nonumber \\= & {} \sum _{k=1}^\infty \frac{-2( \cos (k\pi )-1)}{k\pi } \sin (kt). \end{aligned}$$
(2.41)
Since the terms are O(1/k), this series should converge, since the sum of its coefficients is finite. In Figure 2.10, we show the first 100 terms of the derivative
Figure 2.9:

We illustrate the even and odd \(2\pi \) periodic extensions of \(f(t) = t\) on \([0,\pi ]\) above. Note that while the odd extension \(g_o(t)\) might seem more natural, the even extension \(g_e(t)\) is continuous while the odd extension has a jump discontinuity. As a result, the cosine series converges much faster on \([0,\pi ]\).

of this series. Note that it quite apparently converges for a function which is \(\pm 1\). This is exactly the derivative of the extension \(g_e(t)\) shown in Figure 2.9.

A key difference between the two extensions in Figure 2.9 is that while they both have discontinuities in their derivatives at isolated points, the odd extension has a jump discontinuity in the function. Note that the even extension is \(g_e(t) = |t|\) on \([-\pi ,\pi ]\) and is periodic after that. If we define
$$\begin{aligned} h(t) = \left\{ \begin{array}{c} 1 \text{ if } 2m\pi< t< (2m+1)\pi \\ -1 \text{ if } (2m+1)\pi< t < 2m\pi \end{array}, \right. \end{aligned}$$
(2.42)
then h(t) is the derivative of \(g_e(t)\) except at the points \(k\pi \). More importantly, we can write that
$$\begin{aligned} g_e(t) = \int _0^t h(x) dx ,\end{aligned}$$
(2.43)
and this will be valid for all t, even though the derivative is not defined at \(m\pi \).

Note that we can similarly define the derivative of the odd extension \(g_o(t)\). You will not be able to write an analogue of the equation (2.43), however, due to the jump discontinuity in the function \(g_o(t)\).

We summarize by stating the following

Theorem 2.5.1

(Sine and Cosine Representations) Let f(t) be any function on \(L^2[0,T]\). Then, we can represent f(t) in a series on [0, T] as either
Figure 2.10:

We illustrate the first 100 terms of the series for \(g_e(t)\) and the derivative of that series above 2.41. First note that the series for \(g_e(t)\) converges in the first 10 terms or so, and thus, the 100-term approximation is essentially identical to \(g_e(t)\). Second, notice that the derivative is either approximately 1, or -1, according to the derivative of \(g_e(t)\). In addition, notice the strong Gibbs ringing at the jump discontinuity, and the slow convergence.

$$ f(t) = \frac{a_0}{2} + \sum _{k=1}^\infty a_k\cos \left( \frac{k\pi t}{T}\right) , $$
or
$$ f(t) = \sum _{k=1}^\infty b_k sin\left( \frac{k\pi t}{T}\right) $$
where
$$ a_k = \frac{2}{T} \int _{0}^T f(t)\cos \left( \frac{k\pi t}{T}\right) dt , \text{ and } b_k = \frac{2}{T} \int _{0}^T f(t) \sin \left( \frac{k\pi t}{T} \right) dt. $$

Proof: The proof follows directly from the above discussions and the original Fourier Series theorems.

2.5.1 Problems and Exercises:

  1. 1.

    Calculate the sine and cosine series for \(f(t) = t\) on [0, 1]. Plot the first 30 terms of these series on \([-3,3]\). Estimate the error between both series and the function on [0, 1] after 30 terms. How many more terms of the sine series are necessary to achieve the same error as was achieved with the cosine series and 30 terms?

     
  2. 2.

    Plot the first 30 terms of the derivatives of both the sine and cosine series in the above problem. What do you observe? Do either of them converge, and why?

     
  3. 3.
    Prove that both the cosines \(\{1,\cos (kt)\}_{k=1}^\infty \), and the sines \(\{ \sin (kt)\}_{k=1}^\infty \) are orthogonal on \([0,\pi ]\). Given that
    $$\{1,\cos (kt),\sin (kt)\}_{k=1}^\infty $$
    is complete in \(L^2[-\pi ,\pi ]\) show that both \(\{1,\cos (kt)\}_{k=1}^\infty \) and \(\{ \sin (kt)\}_{k=1}^\infty \) are complete in \(L^2[0,\pi ]\).
     
  4. 4.
    Compute the sine and cosine series for
    $$\begin{aligned} f(t) = \left\{ \begin{array}{l@{\quad }l} t &{} \text{ if } t \le \pi /2 \\ \pi /2 -t &{} \text{ if } t > \pi /2 \end{array}, \right. \end{aligned}$$
    (2.44)
    on \([0,\pi ]\). Which converges faster? Plot the terms. Why do you think this is?
     

2.6 Perhaps Cosine Series Only

In the last section, we showed how a cosine series, which is naturally even, could far more efficiently approximate an odd function. We would like to push this idea one step further, and try to use only cosine series, which are naturally even, to represent an arbitrary function, which is neither odd nor even.

To begin with, remember that we have already constructed a decomposition of an arbitrary function into its even and odd components. Namely, let us refer to our first Fourier Series formula
$$ f(t) = \frac{a_0}{2} + \sum _{k=1}^\infty a_k \cos (kt) + b_k \sin (kt) . $$
The constant and cosine terms are the even portion of the function, and the sine terms are the odd portion of the function. Thus, we know we can write \(f(t) = f_e(t) + f_o(t)\).
Now, we would like to consider writing \(f(t) = f_e(t) + f_o(t)\) in terms of odd and even functions, more directly. Since \(f_e(t) = f_e(-t)\) and \(f_o(t) = -f(-t)\), we have the relations
$$ f_e(t) = \frac{1}{2} ( f(t) + f(-t) ) $$
and
$$ f_o(t) = \frac{1}{2} ( f(t) - f(-t) ). $$
Obviously, \(f(t) = f_e(t) + f_o(t)\) from the definitions above.

What we showed above in the last section is that an even extension of an odd function is more easily representable than the original odd function. Secondly, we have shown that \(t^2\) is more easily representable than t, or that even functions are more easily representable.

The question then is, can’t we make everything even? The answer is yes. We will call this they Compression Series. The algorithm is simple.

Forward Transform:
  1. 1.

    Calculate the cosine series coefficients for \(f_e(t)\).

     
  2. 2.

    Calculate the cosine series coefficients for \(f_o(t)\).

     
  3. 3.

    Figure out how many terms are necessary to represent \(f_e(t)\) and \(f_o(t)\) within a desired accuracy, and keep only these terms.

     

The reconstruction algorithm is similarly simple.

Inverse Transform:
  1. 1.

    Calculate \(f_e(t)\) within the desired accuracy from the above stored coefficients.

     
  2. 2.

    Calculate \(f_o(t)\) within the desired accuracy on \([0,\pi ]\) from the above stored coefficients. Let \(f_o(-t) = - f_o(t)\).

     
  3. 3.

    Calculate \(f(t) = f_e(t) + f_o(t)\) within the desired accuracy with very few coefficients.

     

Theoretically, we have suggested that you can store, or transmit, many fewer coefficients using the above algorithm than blindly using the Fourier Transform. Let us calculate a test example, just to understand what the savings might be. We will use the very simple function \(f(t) = 2 + 3t \), on the interval \([-\pi ,\pi ]\).

The even and odd components of this function are obvious, namely \(f_3(t) = 2\) and \(f_o(t) = 3t\). Now, let us try to represent this function with a minimal number of coefficients, using both the standard Fourier Transform and the cosine series.

The sine series for \(f(t) = t\) on \([-\pi ,\pi ]\) is
$$\begin{aligned} t = \sum _{k=1}^\infty -\frac{2\cos (k\pi )}{k} \sin (kt) . \end{aligned}$$
(2.45)
By linearity, the Fourier Series for our function
$$ f(t) = 2 + 3t = 2 - 3 \sum _{k=1}^\infty \frac{2\cos (k\pi )}{k} \sin (kt) . $$
Let us simplify the discussion by realizing that the only question is “How quickly can be represented the odd portion of the function?”
The cosine series for \(f(t) = t \) is
$$\begin{aligned} t = \frac{\pi }{2} + \sum _{k=1}^\infty \frac{2}{k^2 \pi } ( \cos (k\pi )-1) \cos (kt) , \end{aligned}$$
(2.46)
One of the series decays as O(1/k) and the other as \(O({1}{/}{k^2})\). Thus, the cosine series is more efficient. We illustrate while emphasizing that both obey the isometry rule, or Pythagorean theorem. To do this, we must first rewrite these series in their orthonormal form. This isolates the orthonormal coefficients, and the orthonormal functions, or vectors. The orthonormal forms are
$$\begin{aligned} t = \sum _{k=1}^\infty \left( -\frac{2\cos (k\pi )\sqrt{\pi }}{k}\right) \frac{\sin (kt)}{\sqrt{\pi }} . \end{aligned}$$
(2.47)
and
$$\begin{aligned} t = \frac{\pi ^{3/2}}{\sqrt{2}} \frac{1}{\sqrt{2\pi }} + \sum _{k=1}^\infty \left( \frac{2}{k^2 \pi } ( \cos (k\pi )-1 ) \right) \frac{\cos (kt)}{\sqrt{\pi }} . \end{aligned}$$
(2.48)
The obvious question is “Why do we have to rewrite the equations 2.45 and 2.46 in the more cumbersome forms 2.47 and 2.48?” The answer is that we have to have the orthonormal coefficients. Combining constants in the series makes it more pleasant, but does not allow us to check on the Pythagorean identity.
Let us now examine this and the convergence rates of both series. The norm of the function \(f(t) = t\) is
$$\int _{-\pi }^\pi t^2 dt = \frac{t^3}{3} |_{-\pi }^\pi = \frac{2\pi ^3}{3} \approx 20.67. $$
The Fourier isometry tells us that we also have the strange set of equalities
$$\begin{aligned} \frac{2\pi ^3}{3} = \frac{\pi ^3}{2} + \sum _{k=1}^\infty \left( \frac{2}{k^2 \pi } ( \cos (k\pi )-1 ) \right) ^2 = \sum _{k=1}^\infty \left( -\frac{2\cos (k\pi )\sqrt{\pi }}{k}\right) ^2. \end{aligned}$$
(2.49)
Let us check this numerically. In Figure 2.11, you see the sum of the squared coefficients of the cosine series and the sine series. It is apparent from 2.11 that the cosine series converges much quicker. The absolute numbers can be objectively judged by the relative mean squared error RMSE, which is defined to be
$$ RMSE = \frac{ \Vert f(t) - S_N(f(t)) \Vert ^2}{ \Vert f(t)\Vert ^2}. $$
After 50 terms, the RMSE for the cosine series is 3e-7. The corresponding RMSE for the sine series is .12. After 1000 terms, the sine series has a RMSE of 6e-4. The cosine series achieves this with eight terms. Thus, 1000 coefficients of the sine series are equivalent to eight terms from the cosine series. The savings in computation and storage is very, very significant.

2.6.1 Induced Discontinuities vs. True Discontinuities

The above conversation on using the cosine series focuses on the fact that a Fourier Series is naturally periodic. If the function to be represented is odd, there is an induced jump discontinuity at the endpoint. We have put forward the idea that by changing the odd portion of the function to be even, and then using a cosine representation on it, this can be avoided.
Figure 2.11:

The blue curve above is the sum of the squared coefficients from the cosine series 2.48. The green curve is similarly the sum of the squared coefficients from the sine series 2.47. Note that the cosine series converges much faster.

The question which was not addressed is “What should be done if the function has an actual discontinuity in it?” The answer depends upon the application. If you are using the Fourier Series for computation in a differential equation, you probably do not want to erase the discontinuity, since it is fundamental to the answer. On the other hand, if you are just trying to find efficient storage methods, represent jump discontinuities as exactly what they are. Record the jump discontinuities, and then use the Fourier Transform to represent the smoothed function. This is one of many solutions. A simple example is illustrated in Figure 2.12. The function above is
$$\begin{aligned} f(t) = \left\{ \begin{array}{c} -t+1 \text{ if } |t| < 0 \\ t^2 \text{ if } |t| > 0 \end{array}. \right. \end{aligned}$$
(2.50)
Removing the discontinuity at the origin will result in a much more efficient Fourier Series.

2.6.2 Problems and Exercises:

  1. 1.

    Challenging: Calculate the Fourier Series for the above function. How many terms are necessary to represent it with a RMSE of less than .01?

     
  2. 2.

    Challenging: Remove the discontinuity from the above function, and represent it using only cosine series, and the removed discontinuity. How many terms are necessary to represent it with a RMSE of less than .01?

     
Figure 2.12:

We illustrate a jump discontinuity in the first of the above graphs, and the graph with the jump discontinuity removed below. The standard Fourier Series for this function would converge very slowly with the extension “efficient representation”.

2.7 Gibbs Ringing Phenomenon

Throughout all of our Fourier Series examples, we have noticed a ringing phenomenon, at the discontinuities of the functions. This phenomena is referred to as Gibbs ringing in honor of an American mathematician and physicist who wrote about it in the late 1800s [9]. We will address this phenomena from a more analytical perspective in Chapter  4.

We will look into this more thoroughly from the numerical perspective at this time. We will utilize the Fourier Series of the function \(\chi _{\pi /2}(t)\) which we showed earlier to have the expansion
$$ \chi (t) = \frac{1}{2} + \sum _{k=0}^\infty \frac{2 \sin (k\pi /2)}{k \pi }\cos ( kt ) . $$
Figure 2.13:

We have plotted an approximation to \(\chi (t)\) on \([-\pi ,\pi ]\) above, with 20, 40, and 80 terms. We display this approximation on the interval [1, 2] to get a close-up of the approximations. We notice that the maximum value or peak value of each of the approximations is approximately .1 or so above the desired value of 1. Similarly, the minimum value stays below 0 by a similar amount.

In Figure 2.13, we examine the maximas and minimas of the approximations to this function. We notice that they are all nearly as far above or below the desired function, regardless of the number of terms in the approximation. This is very disturbing, since we have proven that the Fourier Series must converge for all functions in \(L^2[a, b]\). It seems that no matter how many terms are added, there is still a large difference between the maxima, minima, and the desired values of 1 and 0.

There are a number of observations that can be made from Figure 2.13. Putting all of them together allows us to understand what seems like an anomaly which is not consistent with our theorems. Let us note some observations.

  1. 1.

    The maxima and minima of the partial sums do not seem to decrease and approach the final desired value.

     
  2. 2.

    The locations of those maxima and minima seem to get closer to the discontinuity as the number of terms increases.

     
  3. 3.

    The area of the error created by this “ringing” seems to decrease.

     
Let us move from pure speculation based on the graphs, to observation of the numerical facts. The values of the maxima were 1.0899, 1.0896, and 1.0894. The maxima on the right of the discontinuity are located at 1.4140, 1.4920, and 1.5320. Looking at the graphs, it would seem that the maxima will stay at a value around 1.09 and that the location of the maxima will approach \(\pi /2 \approx 1.5708\). Let us test this with 200 terms, illustrated in Figure 2.14. The maximum with 200 terms is located at 1.5550, and its value is 1.0895. This would seem to support our ideas put forward above.
Figure 2.14:

We have plotted an approximation to \(\chi (t)\) on \([-\pi ,\pi ]\) above with 200 terms. We display this approximation on the interval [1, 2] to get a close-up of the approximations. Note that the peak is closer to \(\pi /2\) than those in Figure 2.13, where there were fewer terms in the sum.

We will prove that the above observations demonstrate the truth in Chapter  4. We need more tools to accurately describe this.

2.7.1 Problems and Exercises

  • Gibbs ringing Calculate the Fourier Series for \(\chi _{1/2}(t)\) on \([-1,1]\), and plot the first 50 terms. Find the maximum value of the series with 30, 40, 50, and 100 terms. Guess at the limit of this maximum value. Find the location of the maximum value on the right of 0, with 30, 40, 50, and 100 terms. Where is the maximum value headed? Finally, estimate the squared error \(\int _{-1}^1 |\chi _{1/2}(t) - S_n(\chi _{1/2})(t)|^2dt\) and the series on \([-1,1]\) with the same number of terms (do this numerically).

2.8 Convolution and Correlation

There are two very common operations which come up in association with Fourier Series. They are generally associated with localized averages or operations on a function. We will begin by noting that if \(f(t) \in L^2[-T, T]\), then the extension of f given by \(f = 0\) for \(t>T\) allows us to consider \(f(t) \in L2[-B, B]\) for any \(B>T\). Oftentimes in this section, we will want \(B = 2T\). Now, let us define convolution.

Definition 2.8.1

(Convolution and Correlation) Let \(f, g \in L^2[-T, T]\), and by extension \(L^2[-2*T, 2*T]\). We define the convolution of f and g to be
$$\begin{aligned} f * g (t) = \frac{1}{\sqrt{2T}} \int _{-2T}^{2T} f(x) g(t-x ) dx . \end{aligned}$$
(2.51)
Similarly, we define the correlation of f and g to be
$$\begin{aligned} f \star g (t) = \frac{1}{\sqrt{2T}} \int _{-2T}^{2T} f(x) g(x-t) dx . \end{aligned}$$
(2.52)

We would like to characterize the Fourier Series of these two functions. This leads to the following theorem

Theorem 2.8.1

(Convolution and Correlation Theorem) Let \(f, g \in L^2[-T, T]\). The convolution \(f*g\) will be in \(L^2[-2T, 2T]\). Furthermore, the Fourier Series of \(f*g\) on \(L^2[-T, T]\), which is denoted by \(\mathcal{F}( f*g) \), is expressed simply by
$$\begin{aligned} \mathcal{F}(f*g) = \mathcal{F}(f) \mathcal{F}(g) . \end{aligned}$$
(2.53)
Furthermore, the Fourier Series of the correlation \(f\star g \in L^2[-2T, 2T]\) is given by
$$\begin{aligned} \mathcal{F}( f \star g) = \mathcal{F}(f) \overline{\mathcal{F}(g)} . \end{aligned}$$
(2.54)
Recall that \(\mathcal{F}(f)\) maps the function f to its Fourier coefficients. Thus, the Fourier coefficients of the convolution \(f*g\) can be found by pointwise multiplication of the Fourier coefficients of f and g

Proof: We must first demonstrate that \(f*g \in L^2[-2T, 2T]\). For simplicity’s sake, we will assume that \(T = \pi \), since the general proof follows immediately.

The first step is to realize that this is a series of inner products, and they are therefore bounded by the Cauchy–Schwartz inequality. In other words
$$ f * g (t) = \int _{-2T}^{2T} f(x) g(t-x ) dx = \langle f(x) g(t-x ) \rangle \le \Vert f(x) \Vert _2 \Vert g(t-x) \Vert _2. $$
Another thing to remember is that \(f(x)=0\) for \(|x| > T\) and \(g(t-x) = 0\) for \(|t-x|>T\). Thus, the integral above is zero for \(|x|>T\). In addition, if \(|t|>2T\), it would follow that \(|t-x| > \pi \) so we have that \(f*g(t) = 0\) for \(|t| > 2\pi \). Thus, we have that \(f*g\) is bounded by the Cauchy–Schwartz theorem, and zero outside of \([-2\pi , 2\pi ]\), so the convolution must be square integrable on \([-2\pi , 2\pi ]\), and therefore, \(f*g \in L^2[-2\pi , 2\pi ]\).

We will now prove the second assertion of the theorem. We will do this by simply calculating the Fourier coefficients of the convolution.

The Fourier coefficients are given by
$$\begin{aligned} c_k= & {} \int _{-2\pi }^{2\pi } f*g(t) e^{i (k/2) t } dt = \int _{-2\pi }^{2\pi } \int _{-\infty }^\infty f(x) g(t-x ) dx e^{i (k/2) t } dt \nonumber \\= & {} \int _{-2\pi }^{2\pi } \int _{-\pi }^{\pi } f(x) g(t-x ) dx e^{i (k/2) t } dt \nonumber \\= & {} \int _{-\pi }^{\pi } \int _{-2\pi }^{2\pi } f(x) g(t-x ) e^{i (k/2) t } dt dx \nonumber \\= & {} \int _{-\pi }^{\pi } f(x) \int _{-2\pi }^{2\pi } g(t-x ) e^{i (k/2) (t-x) } dt e^{(k/2)x} dx. \end{aligned}$$
(2.55)
Now, if we let \(u= t-x\), realize that \(g(t-x)\) is only nonzero for \(|u|<\pi \) so we get
$$\begin{aligned} c_k= & {} \int _{-\pi }^{\pi } f(x) \int _{\pi }^\pi g(u ) e^{i (k/2) u } du e^{(k/2)x} dx \nonumber \\= & {} \int _{-\pi }^{\pi } f(x) e^{(k/2)x} dx \int _{\pi }^\pi g(u ) e^{i (k/2) u } du \nonumber \\= & {} \int _{-2\pi }^{2\pi } f(x) e^{(k/2)x} dx \int _{2\pi }^{2\pi } g(x ) e^{i (k/2) x } dx \nonumber \\= & {} \hat{f}(k) \hat{g}(k), \end{aligned}$$
(2.56)
where \(\hat{f}(k)\) and \(\hat{g}(k)\) are the Fourier coefficients on \([-2\pi , 2\pi ]\) for f and g.

2.8.1 A couple of classic examples

One of the problems with Fourier Series is that they are very hard to compute. You have to be able to solve the integral equations, which is not easy for most functions. For this reason, we oftentimes use other means to try to calculate the Fourier Series. The convolution and correlation theorems do provide us with one such method.

Example 1: We begin by picking an example, which we can calculate directly, through the integral equations. We are also able to calculate this example by using the convolution equation. This is a very simple example, but gives a very direct idea of what convolution is.

We start with one of our favorite functions \(f(t) = \chi _\pi (t)\), on \([-\pi ,\pi ]\). We want to consider its extension to the whole real line, namely we want to consider it to be zero outside of \([-\pi ,\pi ]\). Now, we want to consider the convolution \(f*f\). Note that in this case, since \(\chi (t)\) is an even function, convolution and correlation are equal. We will present the result and then leave the calculations as very important exercises.

The convolution \(f*f\) is given by
$$\begin{aligned} f*f(t) = \left\{ \begin{array}{c@{\quad }c} 2\pi -|t| &{} \text{ for } |t| < 2\pi \\ 0 &{} \text{ for } |t| > 2\pi \end{array}. \right. \end{aligned}$$
(2.57)
This is sometimes called a “hat” function and is illustrated in Figure 2.15. An illustrative picture of the “moving average” or convolution process by which the “hat” function was formed is shown in Figure 2.16.

Example 2: We now consider another example. We choose a simple Gaussian which has been corrupted by noise. We then use a window to do a simple moving average of the function to greatly reduce the noise. This is illustrated in Figure 2.17.

2.8.2 Problems and Exercises:

  1. 1.

    Gibbs Ringing Revisited Return to the Fourier Series \(\chi _{\pi /2}(t)\), on \([-\pi ,\pi ]\) which we calculated in 2.8. Calculate a new function, with the Fourier coefficients from 2.8, multiplied by \(exp(-k^2/100)\) for each k. Use 30, 50, and 100 terms. Compare the series with and without the added exponential term. Do you see a reduction in Gibbs ringing? Why?

     
  2. 2.

    Prove that \(f(t) = 2\pi - |t|\) on \([-2\pi , 2\pi ]\) is the convolution of \(\chi _\pi (t) \) against itself, or \(\chi _\pi *\chi _\pi (t) \).

     
  3. 3.

    Calculate the Fourier Series for the hat function \(f(t) = 2\pi - |t|\) on \([-2\pi , 2\pi ]\) in two ways: (a) by calculating its Fourier Series directly. (b) by calculating the Fourier Series for \(\chi _\pi (t) \) and using the convolution theorem.

     
  4. 4.
    Challenging Consider the two functions,
    $$\begin{aligned} f(t) = \left\{ \begin{array}{c@{\quad }c@{\quad }c} \cos (x) &{} \text{ if } &{} |t| < \pi /2 \\ 0 &{} \text{ if } &{}|t| \ge \pi /2 \end{array}. \right. \end{aligned}$$
    (2.58)
    and
    $$\begin{aligned} g(t) = \left\{ \begin{array}{c@{\quad }c@{\quad }c} 0 &{} \text{ if } &{}|t| > .1 \\ -10 &{} \text{ if } &{}t\in [-.1,0] \\ 10 &{} \text{ if } &{}t\in [0,.1]\end{array}. \right. \end{aligned}$$
    (2.59)
    (a) Compute the Fourier Transforms of both functions on \([-\pi ,\pi ]\). (b) Compute the convolution \(f*g\) via the convolution theorem. (c) Plot the first 25 terms of the result. (d) Is the result similar to the derivative of f? (e) Why do you think this is?
     
Figure 2.15:

The convolution of characteristic functions is illustrated in the first graph above.

Figure 2.16:

The convolution process of characteristic functions is illustrated above. One of the functions’ moves is multiplied by the second, and the average of the product is the result.

Figure 2.17:

The idea of using convolution to create a moving average, and denoise a noisy function is illustrated above. The noisy function is multiplied by a simple averaging function, which is 1 on a region, and then an average is done.

2.9 Chapter Project:

  1. 1.

    Compute the Fourier Series for the function \(f(t) = t\) which converges a) on \([-\pi ,\pi ]\) and b) on \([-2\pi , 2\pi ]\). Plot the approximations using 5,10, and 15 terms on \([-4\pi , 4\pi ]\). These are two different series and should look different.

     
  2. 2.

    Figure out the necessary cosine, and sine terms and the appropriate coefficient formulas to approximate a function on the interval [1, 3].

     
  3. 3.

    Using the result from Problem 2 above, calculate the first few terms of the expansion of \(f(t)=t\) on [1, 3]. Plot this result on \([-1,5]\).

     
  4. 4.

    Verify that the Fourier isometry holds on \([-\pi ,\pi ]\) for \(f(t) = t\). To do this, a) calculate the coefficients of the orthogonal Fourier Series from the orthogonal series representation, b) calculate the sum of the squared coefficients, and c) Calculate the norm of the function as \(\int _{-\pi }^\pi |f(t)|^2 dt.\) They must be equal. How many terms in the Fourier Series are necessary to have the isometry be under \(5 \%\)? How many until you are under \(3 \%\), or \(1 \%\)?

     
  5. 5.

    Gibbs ringing: Calculate the Fourier Series for \(\chi _{1/2}(t)\) on \([-1,1]\), and plot the first 50 terms. Find the maximum value of the series with 30, 40, 50, and 100 terms. Guess at the limit of this maximum value. Find the location of the maximum value on the right of 0, with 30, 40, 50, and 100 terms. Where is the maximum value headed? Finally, estimate the squared error \(\int _{-1}^1 |\chi _{1/2}(t) - S_n(\chi _{1/2})(t)|^2dt\) and the series on \([-1,1]\) with the same number of terms (do this numerically).

     
  6. 6.

    Sine and cosine series: Calculate the sine and cosine series for \(f(t) = t\) on [0, 1]. Plot the first 30 terms of these series on \([-3,3]\). Estimate the error between both series and the function on [0, 1] after 30 terms. How many more terms of the sine series are necessary to achieve the same error as was achieved with the cosine series and 30 terms?

     
  7. 7.

    Sine and cosine series: Plot the first 30 terms of the derivatives of both the sine and cosine series in the above problem. What do you observe? Do both of them converge?

     
  8. 8.

    Convolution: Calculate the Fourier Series for the hat function \(f(t) = 2\pi - |t|\) on \([-2\pi , 2\pi ]\) in two ways. a) by calculating its Fourier Series directly. b) by calculating the Fourier Series for \(\chi _\pi (t) \) and using the convolution theorem. They must be equal! Use trig identities...

     

2.10 Summary of Expansions:

  • Fourier expansion for \(f(t) \in L^2[-\pi ,\pi ]\):

    Basic Formula:
    $$ f(t) = \frac{a_0}{2} + \sum _{k=0}^\infty a_k \cos (kt) + b_k \sin (kt) , $$
    where
    $$ a_k = \frac{1}{\pi }\int _{-\pi }^\pi f(t) \cos (kt) dt \text{ and } b_k = \frac{1}{\pi }\int _{-\pi }^\pi f(t) \sin (kt) dt . $$
    Orthonormal Expansion:
    $$ f(t) = a_0 \frac{1}{\sqrt{2\pi }} + \sum _{k=0}^\infty a_k \frac{\cos (kt)}{\sqrt{\pi }} + b_k \frac{\sin (kt)}{\sqrt{\pi }} , $$
    where \( a_0 = 1/\sqrt{2\pi } \int _{-\pi }^\pi f(t) dt \) and for \(k\ge 1\)
    $$ a_k = \int _{-\pi }^\pi f(t) \frac{\cos (kt)}{\sqrt{\pi }} dt \text{ and } b_k = \int _{-\pi }^\pi g(t)\frac{\sin (kt)}{\sqrt{\pi }} dt . $$
  • Fourier expansion for \(f(t) \in L^2[-T, T]\):

    Basic Formula
    $$ f(t) = \frac{a_0}{2} + \frac{1}{T} \sum _{k=0}^\infty a_k \cos \left( \frac{k\pi t}{T}\right) + b_k \sin \left( \frac{k\pi t}{T}\right) , $$
    where
    $$ a_k = \int _{-T}^T f(t) \cos \left( \frac{k\pi t}{T}\right) dt \text{ and } b_k = \int _{-T}^T f(t) sin\left( \frac{k\pi t}{T}\right) dt . $$
    Orthonormal Expansion:
    $$ f(t) = a_0 \frac{1}{\sqrt{2T}} + \sum _{k=0}^\infty a_k \frac{\cos (\frac{k\pi t}{T})}{\sqrt{T}} + b_k \frac{\sin (\frac{k\pi t}{T})}{\sqrt{T}} , $$
    where \( a_0 = 1/\sqrt{2T} \int _{-\pi }^\pi f(t) dt \) and for \(k\ge 1\)
    $$ a_k = \int _{-T}^T f(t) \frac{\cos (\frac{k\pi t}{T})}{\sqrt{T}} dt \text{ and } b_k = \int _{-T}^T f(t)\frac{\sin (\frac{k\pi t}{T})}{\sqrt{T}} dt . $$
  • Fourier expansion for \(f(t) \in L^2[a, b]\): Let \(m = (a+b)/2\), and \(L = (b-a)/2\).
    $$ f(t) = \frac{a_0}{2} + \frac{1}{L} \sum _{k=0}^\infty a_k \cos \left( \frac{k\pi (t-m)}{L}\right) + b_k \sin \left( \frac{k\pi (t-m)}{L}\right) , $$
    where
    $$ a_k = \int _a^b f(t) \cos \left( \frac{k\pi (t-m)}{L}\right) dt \text{ and } b_k = \int _a^b g(t) \sin \left( \frac{k\pi ( t-m)}{L}\right) dt . $$
    Orthonormal Expansion:
    $$ f(t) = a_0 \frac{1}{\sqrt{2L}} + \sum _{k=0}^\infty a_k \frac{\cos \left( \frac{k\pi ( t-m)}{L}\right) }{\sqrt{L}} + b_k \frac{\sin \left( \frac{k\pi (t-m)}{L}\right) }{\sqrt{L}} , $$
    where \( a_0 = 1/\sqrt{2L} \int _{a}^b f(t) dt \) and for \(k\ge 1\)
    $$ a_k = \int _{a}^b f(t) \frac{\cos \left( \frac{k\pi t}{L}\right) }{\sqrt{L}} dt \text{ and } b_k = \int _a^b f(t)\frac{\sin \left( \frac{k\pi t}{L}\right) }{\sqrt{L}} dt . $$
  • Cosine and Sine expansions for \(f(t) \in L^2[0,\pi ]\):

    Basic Formulas:
    $$ f(t) = \frac{a_0}{2} + \sum _{k=0}^\infty a_k \cos (kt) , $$
    and
    $$ f(t) = \sum _{k=0}^\infty b_k \sin (kt) , $$
    where
    $$ a_k = \frac{2}{\pi }\int _{0}^\pi f(t) \cos (kt) dt \text{ and } b_k = \frac{2}{\pi }\int _{0}^\pi f(t) \sin (kt) dt . $$
    Orthonormal Expansion:
    $$ f(t) = a_0 \frac{1}{\sqrt{\pi }} + \sum _{k=0}^\infty a_k \sqrt{\frac{2}{\pi }} \cos (kt) $$
    and
    $$f(t) = \sum _{k=0}^\infty b_k \sqrt{\frac{2}{\pi }} \sin (kt) , $$
    where \( a_0 = 1/\sqrt{\pi } \int _{0}^\pi f(t) dt \) and for \(k\ge 1\)
    $$ a_k = \int _{0}^\pi f(t)\sqrt{\frac{2}{\pi }} \cos (kt) dt \text{ and } b_k = \int _{0}^\pi f(t) \sqrt{\frac{2}{\pi }}\sin (kt) dt . $$

Supplementary material

Supplementary material 1 (avi 663 KB)

Supplementary material 2 (avi 1066 KB)

Supplementary material 3 (avi 2260 KB)

Supplementary material 4 (avi 2260 KB)

Supplementary material 5 (avi 5783 KB)

Supplementary material 6 (avi 1768 KB)

Supplementary material 7 (avi 1750 KB)

Supplementary material 8 (avi 12901 KB)

Supplementary material 9 (avi 7168 KB)

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Department of MathematicsUniversity of FloridaGainesvilleUSA

Personalised recommendations