Chapter 8
Numerical Differentiation
Rendered from the uploaded full LaTeX source with KaTeX. No PDF, no PDF.js, no image extraction, and no raw LaTeX displayed.
After completing this chapter, the reader should be able to:
derive forward, backward, central, and higher-order finite-difference formulas using Taylor expansions;
distinguish truncation error, roundoff error, and total error;
choose a step size that balances discretization and floating-point effects;
construct finite-difference weights from polynomial interpolation;
apply Richardson extrapolation to improve the order of a derivative approximation;
understand one-sided formulas near boundaries;
build first- and second-derivative matrices for finite-difference grids;
derive and use Chebyshev–Gauss–Lobatto differentiation matrices;
compare the algebraic accuracy of finite differences with the spectral accuracy of Chebyshev differentiation;
understand the effect of noise on numerical differentiation;
explain complex-step differentiation and its roundoff advantages;
solve basic, advanced, and research-level exercises on numerical differentiation.
The Numerical Differentiation Problem
The derivative f'(x)=\lim_{h\to0}\frac{f(x+h)-f(x)}{h} is defined by a limiting process. On a computer, however, h cannot be taken arbitrarily small. If h is too large, the approximation suffers from truncation error. If h is too small, subtraction cancellation and roundoff error dominate.
Numerical differentiation is ill-conditioned with respect to noisy data. Small high-frequency perturbations in the function values can produce large perturbations in the derivative.
The simplest formulas are f'(x)\approx\frac{f(x+h)-f(x)}{h}, \qquad f'(x)\approx\frac{f(x)-f(x-h)}{h}, and f'(x)\approx\frac{f(x+h)-f(x-h)}{2h}. The first two are one-sided formulas. The third is the central difference formula.
Taylor Expansions and Difference Formulas
Assume f is sufficiently smooth. Taylor expansion gives f(x+h) = f(x)+hf'(x)+\frac{h^2}{2}f''(x)+\frac{h^3}{6}f^{(3)}(x)+O(h^4), and f(x-h) = f(x)-hf'(x)+\frac{h^2}{2}f''(x)-\frac{h^3}{6}f^{(3)}(x)+O(h^4).
If f\in C^2 near x, then f'(x) = \frac{f(x+h)-f(x)}{h} - \frac{h}{2}f''(\xi) for some \xi between x and x+h. Hence the forward difference is first-order accurate: \frac{f(x+h)-f(x)}{h}=f'(x)+O(h).
Taylor’s theorem with remainder gives f(x+h) = f(x)+hf'(x)+\frac{h^2}{2}f''(\xi). Rearranging, \frac{f(x+h)-f(x)}{h} = f'(x)+\frac{h}{2}f''(\xi). This proves the formula and the O(h) accuracy.
□
If f\in C^3 near x, then f'(x) = \frac{f(x+h)-f(x-h)}{2h} - \frac{h^2}{6}f^{(3)}(\xi) for some \xi\in(x-h,x+h). Hence \frac{f(x+h)-f(x-h)}{2h} = f'(x)+O(h^2).
Taylor’s theorem gives f(x+h)=f(x)+hf'(x)+\frac{h^2}{2}f''(x)+\frac{h^3}{6}f^{(3)}(\xi_+) and f(x-h)=f(x)-hf'(x)+\frac{h^2}{2}f''(x)-\frac{h^3}{6}f^{(3)}(\xi_-). Subtracting, f(x+h)-f(x-h) = 2hf'(x) + \frac{h^3}{6} \left[ f^{(3)}(\xi_+)+f^{(3)}(\xi_-) \right]. Dividing by 2h gives \frac{f(x+h)-f(x-h)}{2h} = f'(x)+O(h^2). The mean-value form of the remainder gives the stated expression for some \xi\in(x-h,x+h).
□
f'(x)=\frac{f(x+h)-f(x)}{h}+O(h), f'(x)=\frac{f(x)-f(x-h)}{h}+O(h), f'(x)=\frac{f(x+h)-f(x-h)}{2h}+O(h^2). The central formula is usually more accurate because the even-order Taylor terms cancel.
Second Derivatives
Adding the Taylor expansions for f(x+h) and f(x-h) gives f(x+h)-2f(x)+f(x-h) = h^2f''(x)+\frac{h^4}{12}f^{(4)}(x)+O(h^6). Thus f''(x) \approx \frac{f(x+h)-2f(x)+f(x-h)}{h^2}.
If f\in C^4, then \frac{f(x+h)-2f(x)+f(x-h)}{h^2} = f''(x)+O(h^2).
Taylor expansion about x gives f(x+h)+f(x-h) = 2f(x)+h^2f''(x)+\frac{h^4}{12}f^{(4)}(x)+O(h^6). Rearranging gives the formula.
□
Higher-Order Finite Differences
Higher-order formulas are obtained by choosing coefficients that make Taylor terms cancel. For example, f'(x) = \frac{-f(x+2h)+8f(x+h)-8f(x-h)+f(x-2h)}{12h} +O(h^4).
If f\in C^5, then \frac{-f(x+2h)+8f(x+h)-8f(x-h)+f(x-2h)}{12h} = f'(x)+O(h^4).
Expand f(x\pm h) and f(x\pm2h) in Taylor series about x. The linear combination -f(x+2h)+8f(x+h)-8f(x-h)+f(x-2h) cancels the constant, second, third, and fourth derivative terms, while the first derivative term equals 12hf'(x). The next nonzero term is proportional to h^5f^{(5)}(x). Division by 12h gives an O(h^4) error.
□
A useful fourth-order formula for the second derivative is f''(x) = \frac{ -f(x+2h)+16f(x+h)-30f(x)+16f(x-h)-f(x-2h) }{12h^2} +O(h^4).
One-Sided Boundary Formulas
Near a boundary, central differences may require values outside the computational domain. One-sided formulas are then needed. A second-order forward formula is f'(x) = \frac{-3f(x)+4f(x+h)-f(x+2h)}{2h} +O(h^2). A second-order backward formula is f'(x) = \frac{3f(x)-4f(x-h)+f(x-2h)}{2h} +O(h^2).
If f\in C^3, then \frac{-3f(x)+4f(x+h)-f(x+2h)}{2h} = f'(x)+O(h^2).
Use Taylor expansions f(x+h)=f(x)+hf'(x)+\frac{h^2}{2}f''(x)+\frac{h^3}{6}f^{(3)}(x)+O(h^4), and f(x+2h)=f(x)+2hf'(x)+2h^2f''(x)+\frac{4h^3}{3}f^{(3)}(x)+O(h^4). Substitution into -3f(x)+4f(x+h)-f(x+2h) cancels the constant and second-derivative terms and leaves 2hf'(x)+O(h^3). Dividing by 2h gives the result.
□
Finite-Difference Weights from Interpolation
Finite-difference formulas can be derived by differentiating an interpolating polynomial. Suppose x_0,\ldots,x_n are distinct nodes and p(t)=\sum_{j=0}^n f(x_j)\ell_j(t) is the Lagrange interpolant. Then f^{(m)}(x_\ast)\approx p^{(m)}(x_\ast) = \sum_{j=0}^n w_j^{(m)}f(x_j), where w_j^{(m)}=\ell_j^{(m)}(x_\ast).
Caption.
Finite-difference weights by polynomial matching
nodes x_0,\ldots,x_n, target x_\ast, derivative order m
Find weights w_0,\ldots,w_n satisfying
\item \sum_{j=0}^n w_j(x_j-x_\ast)^q= \item \begin{cases} \item m!, & q=m,\\ \item 0, & q\ne m, \item \end{cases} \item \qquad \item q=0,\ldots,n. \item
Return w_0,\ldots,w_n
Weights satisfying the moment conditions in the referenced result differentiate exactly every polynomial of degree at most n at x_\ast.
It is enough to test monomials p_q(t)=(t-x_\ast)^q, \qquad q=0,\ldots,n. The exact derivative is p_q^{(m)}(x_\ast) = \begin{cases} m!, & q=m,\\ 0, & q\ne m. \end{cases} The finite-difference formula gives \sum_{j=0}^n w_j(x_j-x_\ast)^q, which matches the same value by the moment conditions. Linearity proves exactness for all polynomials of degree at most n.
□
Richardson Extrapolation
Suppose a derivative approximation has an error expansion D(h)=f'(x)+C h^p+O(h^{p+1}). Then D(h/2)=f'(x)+C\frac{h^p}{2^p}+O(h^{p+1}). Eliminating the leading term gives D_{\mathrm{rich}}(h) = \frac{2^pD(h/2)-D(h)}{2^p-1} = f'(x)+O(h^{p+1}). When the expansion contains only even powers, as for many central formulas, the order improves by two.
If D(h)=D_\ast+C h^p+O(h^{p+q}), with p>0, then \frac{2^pD(h/2)-D(h)}{2^p-1} = D_\ast+O(h^{p+q}).
Substitute the two expansions: D(h)=D_\ast+C h^p+O(h^{p+q}), D(h/2)=D_\ast+C2^{-p} h^p+O(h^{p+q}). Then 2^pD(h/2)-D(h) = (2^p-1)D_\ast+O(h^{p+q}). Dividing by 2^p-1 proves the result.
□
Caption.
Richardson extrapolation for central differences
function f, point x, initial step h, number of levels L
For j=0,\ldots,L:
D_{j,0}\gets \dfrac{f(x+h/2^j)-f(x-h/2^j)}{2h/2^j}
For k=1,\ldots,L:
For j=k,\ldots,L:
D_{j,k}\gets D_{j,k-1}+\dfrac{D_{j,k-1}-D_{j-1,k-1}}{4^k-1}
Return D_{L,L}
Roundoff Error and Optimal Step Size
For the forward difference, the truncation error is O(h). Floating-point evaluation of f(x+h)-f(x) may suffer cancellation. A simple model is E(h) \approx C_1h+C_2\frac{\epsmach}{h}. The first term is truncation error. The second term is roundoff amplification.
Minimizing this model gives h_{\mathrm{opt}} \approx \sqrt{\epsmach}. For the central difference, E(h)\approx C_1h^2+C_2\frac{\epsmach}{h}, and therefore h_{\mathrm{opt}}\approx \epsmach^{1/3}.
Suppose E(h)=Ah^p+B\frac{\epsmach}{h}, \qquad A>0,\quad B>0. Then the minimizing step size satisfies h_{\mathrm{opt}} = \left( \frac{B\epsmach}{pA} \right)^{1/(p+1)}.
Differentiate: E'(h)=pAh^{p-1}-B\epsmach h^{-2}. Setting E'(h)=0 gives pAh^{p+1}=B\epsmach. Thus h_{\mathrm{opt}} = \left( \frac{B\epsmach}{pA} \right)^{1/(p+1)}.
□
Complex-Step Differentiation
For real analytic f, use the Taylor expansion f(x+ih) = f(x)+ihf'(x)-\frac{h^2}{2}f''(x)-i\frac{h^3}{6}f^{(3)}(x)+O(h^4). Taking imaginary parts, f'(x) = \frac{\operatorname{Im} f(x+ih)}{h} +O(h^2).
The complex-step formula does not subtract nearly equal real numbers. Therefore it avoids the catastrophic cancellation that limits ordinary finite differences.
If f is analytic near x, then \frac{\operatorname{Im} f(x+ih)}{h} = f'(x)+O(h^2).
The complex Taylor expansion gives f(x+ih) = f(x)+ihf'(x)-\frac{h^2}{2}f''(x)-i\frac{h^3}{6}f^{(3)}(x)+O(h^4). Taking imaginary parts and dividing by h gives \frac{\operatorname{Im}f(x+ih)}{h} = f'(x)-\frac{h^2}{6}f^{(3)}(x)+O(h^4). Thus the error is O(h^2).
□
Differentiation Matrices on Uniform Grids
Let x_i=a+ih, \qquad i=0,\ldots,N, \qquad h=\frac{b-a}{N}. If \mathbf f=(f(x_0),\ldots,f(x_N))^T, then a differentiation matrix D approximates \mathbf f'\approx D\mathbf f. For interior central differences, (D\mathbf f)_i = \frac{f_{i+1}-f_{i-1}}{2h}, \qquad i=1,\ldots,N-1.
A second-derivative matrix approximates \mathbf f''\approx D_2\mathbf f, with (D_2\mathbf f)_i = \frac{f_{i+1}-2f_i+f_{i-1}}{h^2}.
Chebyshev Nodes
Finite differences are local. Chebyshev differentiation is global and spectral. On [-1,1], the Chebyshev–Gauss–Lobatto nodes are x_j=\cos\left(\frac{\pi j}{N}\right), \qquad j=0,\ldots,N. They cluster near the endpoints. This clustering controls interpolation oscillations and makes global polynomial differentiation stable compared with equispaced global interpolation.
Chebyshev Differentiation Matrix
Let x_j=\cos\left(\frac{\pi j}{N}\right), \qquad j=0,\ldots,N. Define c_j= \begin{cases} 2, & j=0 \text{ or } j=N,\\ 1, & 1\le j\le N-1. \end{cases} The Chebyshev first-differentiation matrix D=(D_{ij})_{i,j=0}^{N} on [-1,1] is given by D_{ij} = \frac{c_i}{c_j} \frac{(-1)^{i+j}}{x_i-x_j}, \qquad i\ne j, and D_{ii} = -\frac{x_i}{2(1-x_i^2)}, \qquad 1\le i\le N-1, with endpoint entries D_{00}=\frac{2N^2+1}{6}, \qquad D_{NN}=-\frac{2N^2+1}{6}. Then f'(x_i)\approx \sum_{j=0}^N D_{ij}f(x_j).
For an interval [a,b], use the affine map x=\frac{2t-(a+b)}{b-a}. Since \frac{d}{dt}=\frac{2}{b-a}\frac{d}{dx}, the physical differentiation matrix is D^{[a,b]}=\frac{2}{b-a}D. Higher derivatives are approximated by powers: f^{(m)}(\mathbf x)\approx D^m\mathbf f, and on [a,b], D_m^{[a,b]}=\left(\frac{2}{b-a}\right)^mD^m.
Caption.
Chebyshev first-differentiation matrix
polynomial degree N
For j=0,\ldots,N:
x_j\gets \cos(\pi j/N)
c_j\gets 2 if j=0 or j=N, otherwise c_j\gets1
For i=0,\ldots,N:
For j=0,\ldots,N:
If i\ne j:
D_{ij}\gets \dfrac{c_i}{c_j}\dfrac{(-1)^{i+j}}{x_i-x_j}
For i=1,\ldots,N-1:
D_{ii}\gets -x_i/[2(1-x_i^2)]
D_{00}\gets (2N^2+1)/6
D_{NN}\gets -(2N^2+1)/6
Return nodes x_j and matrix D
Let p be a polynomial of degree at most N. If \mathbf p=(p(x_0),\ldots,p(x_N))^T at the Chebyshev–Gauss–Lobatto nodes, then (D\mathbf p)_i=p'(x_i), \qquad i=0,\ldots,N.
Let \ell_j(x) be the Lagrange cardinal polynomial associated with x_j. The interpolating polynomial is p(x)=\sum_{j=0}^N p(x_j)\ell_j(x), because p already has degree at most N. Differentiating, p'(x_i)=\sum_{j=0}^N p(x_j)\ell_j'(x_i). The entries of the Chebyshev differentiation matrix are precisely D_{ij}=\ell_j'(x_i). Therefore p'(x_i)=(D\mathbf p)_i.
□
Chebyshev Accuracy versus Finite-Difference Accuracy
Finite-difference methods approximate derivatives locally. A p-th order finite difference formula typically satisfies \max_i |f'(x_i)-(D_h\mathbf f)_i| = O(h^p) for smooth functions, where h\sim N^{-1}. Thus finite-difference convergence is algebraic: O(N^{-p}).
Chebyshev differentiation is global. For analytic functions, Chebyshev interpolation errors often decay geometrically: \|f-p_N\|_\infty \le C\rho^{-N}, \qquad \rho>1. Derivative errors also decay rapidly, often spectrally, until roundoff error or resolution limits are reached.
If f is analytic in a complex neighborhood of [-1,1], then Chebyshev differentiation converges faster than any algebraic power N^{-p}, up to constants depending on p and on the domain of analyticity.
Analyticity implies rapid decay of Chebyshev coefficients. If f(x)=\sum_{k=0}^{\infty} a_kT_k(x), then, for functions analytic in a Bernstein ellipse of parameter \rho>1, |a_k|\le C\rho^{-k}. Truncating the Chebyshev series at degree N therefore gives a geometrically small interpolation error. Differentiation multiplies the k-th spectral component by a factor growing algebraically in k, but algebraic growth is dominated by geometric decay. Hence derivative errors still decrease faster than any fixed algebraic order N^{-p}, until finite precision dominates.
□
- Finite differences.
-
Local, sparse, simple, geometry-friendly, and efficient for large PDE grids. Accuracy is usually algebraic: O(h^p).
- Chebyshev differentiation.
-
Global, dense, highly accurate for smooth or analytic functions on simple intervals. For analytic functions it may exhibit spectral or near-exponential convergence.
- Finite-difference weakness.
-
High order needs wider stencils; boundary closure is delicate; numerical differentiation amplifies noise.
- Chebyshev weakness.
-
Matrices are dense; endpoint clustering can produce severe time-step restrictions in time-dependent PDEs; nonsmooth functions lose spectral convergence.
- Practical rule.
-
Use finite differences for large local PDE discretizations and irregular domains. Use Chebyshev differentiation for smooth problems on simple intervals when high accuracy per degree of freedom is desired.
Chebyshev Differentiation for Boundary-Value Problems
For a boundary-value problem such as -u''(x)=f(x), \qquad -1<x<1, \qquad u(-1)=u(1)=0, one forms the Chebyshev second-derivative matrix D^{(2)}=D^2. Boundary conditions are imposed by replacing boundary equations or by extracting interior rows and columns. If \mathbf u=(u(x_0),\ldots,u(x_N))^T, then the interior collocation equations are -D^{(2)}_{I,I}\mathbf u_I=\mathbf f_I, where I=\{1,\ldots,N-1\}.
Noise Amplification
Differentiation amplifies noise. If \tilde f(x_i)=f(x_i)+\eta_i, then applying a differentiation matrix gives D\tilde{\mathbf f}=D\mathbf f+D\boldsymbol\eta. Even when \boldsymbol\eta is small, D\boldsymbol\eta may be large.
Numerical differentiation should not be applied blindly to noisy measurements. Regularization, smoothing, model fitting, or weak formulations may be necessary.
Common approaches include: \text{smoothing splines}, \qquad \text{Savitzky--Golay filters}, \qquad \text{Tikhonov regularization}, \qquad \text{total variation methods}.
Savitzky–Golay Differentiation
Savitzky–Golay differentiation fits a local polynomial to data in a moving window and differentiates the fitted polynomial. This can be viewed as a local least-squares finite-difference method. It is useful when data are noisy but locally smooth.
Automatic Differentiation versus Numerical Differentiation
Numerical differentiation approximates derivatives from function values. Automatic differentiation applies the chain rule to the computational graph of a program. In exact arithmetic, automatic differentiation gives exact derivatives of the implemented function, not finite-difference approximations.
- Forward difference.
-
First order, simple, but sensitive to cancellation.
- Central difference.
-
Second order, usually a much better default finite-difference formula.
- High-order finite differences.
-
More accurate for smooth data, but use wider stencils and can be more sensitive to noise.
- Richardson extrapolation.
-
Improves order when the asymptotic error expansion is reliable.
- Complex-step differentiation.
-
Excellent for analytic real-valued functions extended to complex arguments. Avoids subtractive cancellation.
- Chebyshev differentiation.
-
Spectral accuracy for analytic functions on simple intervals, but global and dense.
- Automatic differentiation.
-
Best when derivatives of a known computational program are needed and the program is compatible with AD.
- Regularized differentiation.
-
Essential when data are noisy or ill-conditioned.
Exercises
The following exercise bank is intentionally large. Basic problems test definitions, computations, and essential formulas. Starred exercises require proofs, careful error analysis, or spectral approximation arguments. Problems marked \star, \star\star, and \star\star\star are progressively harder.
Basic problems
Derive the forward difference formula for f'(x) and show that it is first-order accurate.
Derive the backward difference formula for f'(x) and state its order of accuracy.
Derive the central difference formula f'(x)\approx \frac{f(x+h)-f(x-h)}{2h}.
Prove that the central difference formula has error O(h^2).
Derive f''(x)\approx \frac{f(x+h)-2f(x)+f(x-h)}{h^2}.
Derive the second-order forward formula f'(x)\approx \frac{-3f(x)+4f(x+h)-f(x+2h)}{2h}.
Verify the fourth-order central formula for f'(x).
Explain how Richardson extrapolation cancels the leading error term.
Apply one Richardson extrapolation step to the central difference formula.
Explain the model E(h)\approx C_1h^p+C_2\epsmach/h.
Using the model E(h)=h^2+\epsmach/h, derive the optimal scaling of h.
Derive f'(x)\approx \frac{\operatorname{Im} f(x+ih)}{h}.
Write the first-derivative central-difference matrix on a uniform grid with five points, using one-sided boundary formulas.
Compute the Chebyshev–Gauss–Lobatto nodes for N=4.
Explain why Chebyshev nodes cluster near -1 and 1.
State the formula for the off-diagonal entries of the Chebyshev differentiation matrix.
State the formula for the diagonal entries of the Chebyshev differentiation matrix.
If D differentiates on [-1,1], derive the scaling for differentiation on [a,b].
State two advantages of finite differences and two advantages of Chebyshev differentiation.
Explain why differentiation amplifies noise.
Intermediate problems \star
Prove the referenced result.
Prove the referenced result.
Prove the referenced result.
Prove the referenced result.
Prove the referenced result.
Prove the referenced result.
Derive the derivative formula at x_0 using arbitrary distinct nodes x_0,x_1,x_2.
Construct a second-order derivative approximation on an unequally spaced grid.
Prove the referenced result.
Build a Richardson extrapolation table for the central difference formula.
Prove the referenced result.
Use the roundoff model to derive h_{\mathrm{opt}}\sim\epsmach^{1/2} for forward differences.
Use the roundoff model to derive h_{\mathrm{opt}}\sim\epsmach^{1/3} for central differences.
Prove the referenced result.
Give examples of functions or programs for which complex-step differentiation is not directly applicable.
Construct first- and second-derivative matrices on a uniform grid with N+1 points.
Explain why boundary closure can reduce the global order of a finite-difference method.
Prove the referenced result.
Derive D^{[a,b]}=\frac{2}{b-a}D.
Show how to approximate f'' at Chebyshev nodes using D^2.
Explain why a p-th order finite-difference formula gives O(N^{-p}) convergence on a uniform grid.
Explain why Chebyshev differentiation can converge faster than any algebraic power for analytic functions.
Explain what happens to Chebyshev differentiation when f is not smooth.
Given noisy data \tilde{\mathbf f}=\mathbf f+\eta, show that the derivative error contains D\eta.
Explain Savitzky–Golay differentiation as local polynomial least squares.
Advanced problems \star\star
Derive Fornberg’s recursive algorithm for finite-difference weights on arbitrary nodes.
Construct a sixth-order central finite-difference formula for f'(x).
Derive a fourth-order compact finite-difference formula for the first derivative.
Analyze the modified wavenumber of second- and fourth-order central differences.
Study dispersion error of finite-difference differentiation for oscillatory functions.
Explain why differentiation is an unbounded operator from C^0 to C^0.
Formulate numerical differentiation as a Tikhonov-regularized inverse problem.
Study total-variation-regularized differentiation for noisy data.
Derive the off-diagonal and diagonal entries of the Chebyshev differentiation matrix from Lagrange cardinal functions.
Derive D_{00}=\frac{2N^2+1}{6}, \qquad D_{NN}=-\frac{2N^2+1}{6}.
Derive a recurrence for differentiating a Chebyshev series in coefficient space.
Derive Chebyshev differentiation formulas using barycentric interpolation weights.
Design a numerical experiment comparing second-order FDM, fourth-order FDM, and Chebyshev differentiation for f(x)=e^x on [-1,1].
Relate the rate of Chebyshev convergence to the distance of complex singularities from [-1,1].
Explain why equispaced global polynomial differentiation can be unstable and how Chebyshev nodes help.
Investigate how norms and spectral properties of Chebyshev differentiation matrices grow with N.
Use Chebyshev differentiation to discretize -u''(x)=f(x),\qquad u(-1)=u(1)=0.
Explain why Chebyshev endpoint clustering is useful for boundary layers.
Discuss aliasing in spectral differentiation and its effect on nonlinear problems.
Design a spectral filter for reducing noise in Chebyshev differentiation.
Construct tensor-product differentiation matrices for a rectangular domain.
Compare memory and computational cost of FDM differentiation matrices and Chebyshev differentiation matrices.
Compare finite differences, complex-step differentiation, symbolic differentiation, and automatic differentiation.
Analyze the error f'-p_N' when p_N is an interpolation polynomial.
Explain the role of Lebesgue constants in global polynomial differentiation.
Research-level problems \star\star\star
Prove a rigorous Chebyshev derivative error bound for analytic functions in a Bernstein ellipse.
Derive Chebyshev differentiation error estimates for functions of finite Sobolev regularity.
Study Gegenbauer reconstruction for recovering spectral accuracy near discontinuities.
Investigate spectral viscosity methods for stabilizing spectral differentiation in nonsmooth problems.
Study summation-by-parts finite-difference operators and their role in stable PDE discretizations.
Analyze compact finite-difference schemes using Padé approximations and modified wavenumbers.
Develop a statistical error model for numerical differentiation from noisy data.
Study Gaussian-process regression as a method for differentiating noisy observations.
Develop an L^1-regularized differentiation method for signals with sparse derivative structure.
Compare classical numerical differentiation with fractional differentiation matrices for nonlocal operators.
Use Chebyshev differentiation matrices to discretize a nonlinear boundary-value problem and analyze Newton linearization.
Analyze severe time-step restrictions caused by Chebyshev differentiation matrices in explicit time stepping.
Develop a matrix-free Chebyshev spectral differentiation algorithm using transforms.
Study ultraspherical spectral methods and compare their conditioning with classical Chebyshev collocation.
Design an adaptive numerical differentiation method that chooses both stencil width and step size.
Develop interval-arithmetic certificates for finite-difference derivative approximations.
Design reproducible experiments comparing FDM and Chebyshev differentiation across smooth, analytic, nonsmooth, and noisy functions.
Use tensor-product Chebyshev differentiation matrices to discretize a two-dimensional elliptic PDE and analyze the resulting linear system.
Design a hybrid method using finite differences in one coordinate and Chebyshev differentiation in another.
Compare finite differences, automatic differentiation, and spectral differentiation for physics-informed neural networks.
Exercise solutions
The solutions below are converted from the uploaded LaTeX solution file. Open an accordion to view the full solution.
This chapter gives detailed worked solutions for the exercises in Chapter 8. Each solution includes the problem formulation, the method, the mathematical derivation, the conclusion, and a diagnostic comment.
Exercise 8.1
Problem formulation.
Derive the forward difference formula for f'(x) and show that it is first-order accurate.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.2
Problem formulation.
Derive the backward difference formula for f'(x) and state its order of accuracy.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.3
Problem formulation.
Derive the central difference formula f'(x)\approx \frac{f(x+h)-f(x-h)}{2h}.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Taylor expand: f(x+h)=f(x)+hf'(x)+\frac{h^2}{2}f''(x)+\frac{h^3}{6}f'''(x)+O(h^4), f(x-h)=f(x)-hf'(x)+\frac{h^2}{2}f''(x)-\frac{h^3}{6}f'''(x)+O(h^4). Subtracting, f(x+h)-f(x-h)=2hf'(x)+\frac{h^3}{3}f'''(x)+O(h^5). Thus f'(x)=\frac{f(x+h)-f(x-h)}{2h}+O(h^2).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.4
Problem formulation.
Prove that the central difference formula has error O(h^2).
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Taylor expand: f(x+h)=f(x)+hf'(x)+\frac{h^2}{2}f''(x)+\frac{h^3}{6}f'''(x)+O(h^4), f(x-h)=f(x)-hf'(x)+\frac{h^2}{2}f''(x)-\frac{h^3}{6}f'''(x)+O(h^4). Subtracting, f(x+h)-f(x-h)=2hf'(x)+\frac{h^3}{3}f'''(x)+O(h^5). Thus f'(x)=\frac{f(x+h)-f(x-h)}{2h}+O(h^2).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.5
Problem formulation.
Derive f''(x)\approx \frac{f(x+h)-2f(x)+f(x-h)}{h^2}.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Add the Taylor expansions: f(x+h)+f(x-h) = 2f(x)+h^2f''(x)+\frac{h^4}{12}f^{(4)}(x)+O(h^6). Therefore f''(x)= \frac{f(x-h)-2f(x)+f(x+h)}{h^2}+O(h^2).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.6
Problem formulation.
Derive the second-order forward formula f'(x)\approx \frac{-3f(x)+4f(x+h)-f(x+2h)}{2h}.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Add the Taylor expansions: f(x+h)+f(x-h) = 2f(x)+h^2f''(x)+\frac{h^4}{12}f^{(4)}(x)+O(h^6). Therefore f''(x)= \frac{f(x-h)-2f(x)+f(x+h)}{h^2}+O(h^2).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.7
Problem formulation.
Verify the fourth-order central formula for f'(x).
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Taylor expand: f(x+h)=f(x)+hf'(x)+\frac{h^2}{2}f''(x)+\frac{h^3}{6}f'''(x)+O(h^4), f(x-h)=f(x)-hf'(x)+\frac{h^2}{2}f''(x)-\frac{h^3}{6}f'''(x)+O(h^4). Subtracting, f(x+h)-f(x-h)=2hf'(x)+\frac{h^3}{3}f'''(x)+O(h^5). Thus f'(x)=\frac{f(x+h)-f(x-h)}{2h}+O(h^2).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.8
Problem formulation.
Explain how Richardson extrapolation cancels the leading error term.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Assume D(h)=D+Ch^p+O(h^{p+1}). Then D(h/2)=D+C\frac{h^p}{2^p}+O(h^{p+1}). Eliminate C h^p: 2^pD(h/2)-D(h)=(2^p-1)D+O(h^{p+1}). Hence D_{\rm rich} = \frac{2^pD(h/2)-D(h)}{2^p-1} = D+O(h^{p+1}).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.9
Problem formulation.
Apply one Richardson extrapolation step to the central difference formula.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Taylor expand: f(x+h)=f(x)+hf'(x)+\frac{h^2}{2}f''(x)+\frac{h^3}{6}f'''(x)+O(h^4), f(x-h)=f(x)-hf'(x)+\frac{h^2}{2}f''(x)-\frac{h^3}{6}f'''(x)+O(h^4). Subtracting, f(x+h)-f(x-h)=2hf'(x)+\frac{h^3}{3}f'''(x)+O(h^5). Thus f'(x)=\frac{f(x+h)-f(x-h)}{2h}+O(h^2).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.10
Problem formulation.
Explain the model E(h)\approx C_1h^p+C_2\epsmach/h.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
For the forward difference, D_hf=\frac{f(x+h)-f(x)}{h}. The truncation error is O(h), while the subtraction and division amplify roundoff like O(u/h). Hence E(h)\approx C_1h+C_2\frac{u}{h}. Minimize: E'(h)=C_1-C_2\frac{u}{h^2}=0. Therefore h_{\rm opt}\approx\sqrt{\frac{C_2u}{C_1}}, so the practical scale is h_{\rm opt}=O(\sqrt u).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
Distinguish forward error from backward error; a small residual does not imply a small forward error when the problem is ill-conditioned.
Exercise 8.11
Problem formulation.
Using the model E(h)=h^2+\epsmach/h, derive the optimal scaling of h.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.12
Problem formulation.
Derive f'(x)\approx \frac{\operatorname{Im} f(x+ih)}{h}.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.13
Problem formulation.
Write the first-derivative central-difference matrix on a uniform grid with five points, using one-sided boundary formulas.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Taylor expand: f(x+h)=f(x)+hf'(x)+\frac{h^2}{2}f''(x)+\frac{h^3}{6}f'''(x)+O(h^4), f(x-h)=f(x)-hf'(x)+\frac{h^2}{2}f''(x)-\frac{h^3}{6}f'''(x)+O(h^4). Subtracting, f(x+h)-f(x-h)=2hf'(x)+\frac{h^3}{3}f'''(x)+O(h^5). Thus f'(x)=\frac{f(x+h)-f(x-h)}{2h}+O(h^2).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.14
Problem formulation.
Compute the Chebyshev–Gauss–Lobatto nodes for N=4.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.15
Problem formulation.
Explain why Chebyshev nodes cluster near -1 and 1.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.16
Problem formulation.
State the formula for the off-diagonal entries of the Chebyshev differentiation matrix.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.17
Problem formulation.
State the formula for the diagonal entries of the Chebyshev differentiation matrix.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.18
Problem formulation.
If D differentiates on [-1,1], derive the scaling for differentiation on [a,b].
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.19
Problem formulation.
State two advantages of finite differences and two advantages of Chebyshev differentiation.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.20
Problem formulation.
Explain why differentiation amplifies noise.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.21
Problem formulation.
Prove the referenced result.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.22
Problem formulation.
Prove the referenced result.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Taylor expand: f(x+h)=f(x)+hf'(x)+\frac{h^2}{2}f''(x)+\frac{h^3}{6}f'''(x)+O(h^4), f(x-h)=f(x)-hf'(x)+\frac{h^2}{2}f''(x)-\frac{h^3}{6}f'''(x)+O(h^4). Subtracting, f(x+h)-f(x-h)=2hf'(x)+\frac{h^3}{3}f'''(x)+O(h^5). Thus f'(x)=\frac{f(x+h)-f(x-h)}{2h}+O(h^2).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.23
Problem formulation.
Prove the referenced result.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Taylor expand: f(x+h)=f(x)+hf'(x)+\frac{h^2}{2}f''(x)+\frac{h^3}{6}f'''(x)+O(h^4), f(x-h)=f(x)-hf'(x)+\frac{h^2}{2}f''(x)-\frac{h^3}{6}f'''(x)+O(h^4). Subtracting, f(x+h)-f(x-h)=2hf'(x)+\frac{h^3}{3}f'''(x)+O(h^5). Thus f'(x)=\frac{f(x+h)-f(x-h)}{2h}+O(h^2).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.24
Problem formulation.
Prove the referenced result.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Taylor expand: f(x+h)=f(x)+hf'(x)+\frac{h^2}{2}f''(x)+\frac{h^3}{6}f'''(x)+O(h^4), f(x-h)=f(x)-hf'(x)+\frac{h^2}{2}f''(x)-\frac{h^3}{6}f'''(x)+O(h^4). Subtracting, f(x+h)-f(x-h)=2hf'(x)+\frac{h^3}{3}f'''(x)+O(h^5). Thus f'(x)=\frac{f(x+h)-f(x-h)}{2h}+O(h^2).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.25
Problem formulation.
Prove the referenced result.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Add the Taylor expansions: f(x+h)+f(x-h) = 2f(x)+h^2f''(x)+\frac{h^4}{12}f^{(4)}(x)+O(h^6). Therefore f''(x)= \frac{f(x-h)-2f(x)+f(x+h)}{h^2}+O(h^2).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.26
Problem formulation.
Prove the referenced result.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.27
Problem formulation.
Derive the derivative formula at x_0 using arbitrary distinct nodes x_0,x_1,x_2.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.28
Problem formulation.
Construct a second-order derivative approximation on an unequally spaced grid.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Add the Taylor expansions: f(x+h)+f(x-h) = 2f(x)+h^2f''(x)+\frac{h^4}{12}f^{(4)}(x)+O(h^6). Therefore f''(x)= \frac{f(x-h)-2f(x)+f(x+h)}{h^2}+O(h^2).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.29
Problem formulation.
Prove the referenced result.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Assume D(h)=D+Ch^p+O(h^{p+1}). Then D(h/2)=D+C\frac{h^p}{2^p}+O(h^{p+1}). Eliminate C h^p: 2^pD(h/2)-D(h)=(2^p-1)D+O(h^{p+1}). Hence D_{\rm rich} = \frac{2^pD(h/2)-D(h)}{2^p-1} = D+O(h^{p+1}).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.30
Problem formulation.
Build a Richardson extrapolation table for the central difference formula.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Taylor expand: f(x+h)=f(x)+hf'(x)+\frac{h^2}{2}f''(x)+\frac{h^3}{6}f'''(x)+O(h^4), f(x-h)=f(x)-hf'(x)+\frac{h^2}{2}f''(x)-\frac{h^3}{6}f'''(x)+O(h^4). Subtracting, f(x+h)-f(x-h)=2hf'(x)+\frac{h^3}{3}f'''(x)+O(h^5). Thus f'(x)=\frac{f(x+h)-f(x-h)}{2h}+O(h^2).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.31
Problem formulation.
Prove the referenced result.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.32
Problem formulation.
Use the roundoff model to derive h_{\mathrm{opt}}\sim\epsmach^{1/2} for forward differences.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
For the forward difference, D_hf=\frac{f(x+h)-f(x)}{h}. The truncation error is O(h), while the subtraction and division amplify roundoff like O(u/h). Hence E(h)\approx C_1h+C_2\frac{u}{h}. Minimize: E'(h)=C_1-C_2\frac{u}{h^2}=0. Therefore h_{\rm opt}\approx\sqrt{\frac{C_2u}{C_1}}, so the practical scale is h_{\rm opt}=O(\sqrt u).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
Distinguish forward error from backward error; a small residual does not imply a small forward error when the problem is ill-conditioned.
Exercise 8.33
Problem formulation.
Use the roundoff model to derive h_{\mathrm{opt}}\sim\epsmach^{1/3} for central differences.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Taylor expand: f(x+h)=f(x)+hf'(x)+\frac{h^2}{2}f''(x)+\frac{h^3}{6}f'''(x)+O(h^4), f(x-h)=f(x)-hf'(x)+\frac{h^2}{2}f''(x)-\frac{h^3}{6}f'''(x)+O(h^4). Subtracting, f(x+h)-f(x-h)=2hf'(x)+\frac{h^3}{3}f'''(x)+O(h^5). Thus f'(x)=\frac{f(x+h)-f(x-h)}{2h}+O(h^2).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
Distinguish forward error from backward error; a small residual does not imply a small forward error when the problem is ill-conditioned.
Exercise 8.34
Problem formulation.
Prove the referenced result.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.35
Problem formulation.
Give examples of functions or programs for which complex-step differentiation is not directly applicable.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.36
Problem formulation.
Construct first- and second-derivative matrices on a uniform grid with N+1 points.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Add the Taylor expansions: f(x+h)+f(x-h) = 2f(x)+h^2f''(x)+\frac{h^4}{12}f^{(4)}(x)+O(h^6). Therefore f''(x)= \frac{f(x-h)-2f(x)+f(x+h)}{h^2}+O(h^2).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.37
Problem formulation.
Explain why boundary closure can reduce the global order of a finite-difference method.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.38
Problem formulation.
Prove the referenced result.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.39
Problem formulation.
Derive D^{[a,b]}=\frac{2}{b-a}D.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.40
Problem formulation.
Show how to approximate f'' at Chebyshev nodes using D^2.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Add the Taylor expansions: f(x+h)+f(x-h) = 2f(x)+h^2f''(x)+\frac{h^4}{12}f^{(4)}(x)+O(h^6). Therefore f''(x)= \frac{f(x-h)-2f(x)+f(x+h)}{h^2}+O(h^2).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.41
Problem formulation.
Explain why a p-th order finite-difference formula gives O(N^{-p}) convergence on a uniform grid.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.42
Problem formulation.
Explain why Chebyshev differentiation can converge faster than any algebraic power for analytic functions.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.43
Problem formulation.
Explain what happens to Chebyshev differentiation when f is not smooth.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.44
Problem formulation.
Given noisy data \tilde{\mathbf f}=\mathbf f+\eta, show that the derivative error contains D\eta.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.45
Problem formulation.
Explain Savitzky–Golay differentiation as local polynomial least squares.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.46
Problem formulation.
Derive Fornberg’s recursive algorithm for finite-difference weights on arbitrary nodes.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.47
Problem formulation.
Construct a sixth-order central finite-difference formula for f'(x).
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Taylor expand: f(x+h)=f(x)+hf'(x)+\frac{h^2}{2}f''(x)+\frac{h^3}{6}f'''(x)+O(h^4), f(x-h)=f(x)-hf'(x)+\frac{h^2}{2}f''(x)-\frac{h^3}{6}f'''(x)+O(h^4). Subtracting, f(x+h)-f(x-h)=2hf'(x)+\frac{h^3}{3}f'''(x)+O(h^5). Thus f'(x)=\frac{f(x+h)-f(x-h)}{2h}+O(h^2).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.48
Problem formulation.
Derive a fourth-order compact finite-difference formula for the first derivative.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.49
Problem formulation.
Analyze the modified wavenumber of second- and fourth-order central differences.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Taylor expand: f(x+h)=f(x)+hf'(x)+\frac{h^2}{2}f''(x)+\frac{h^3}{6}f'''(x)+O(h^4), f(x-h)=f(x)-hf'(x)+\frac{h^2}{2}f''(x)-\frac{h^3}{6}f'''(x)+O(h^4). Subtracting, f(x+h)-f(x-h)=2hf'(x)+\frac{h^3}{3}f'''(x)+O(h^5). Thus f'(x)=\frac{f(x+h)-f(x-h)}{2h}+O(h^2).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.50
Problem formulation.
Study dispersion error of finite-difference differentiation for oscillatory functions.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.51
Problem formulation.
Explain why differentiation is an unbounded operator from C^0 to C^0.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.52
Problem formulation.
Formulate numerical differentiation as a Tikhonov-regularized inverse problem.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.53
Problem formulation.
Study total-variation-regularized differentiation for noisy data.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.54
Problem formulation.
Derive the off-diagonal and diagonal entries of the Chebyshev differentiation matrix from Lagrange cardinal functions.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.55
Problem formulation.
Derive D_{00}=\frac{2N^2+1}{6}, \qquad D_{NN}=-\frac{2N^2+1}{6}.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.56
Problem formulation.
Derive a recurrence for differentiating a Chebyshev series in coefficient space.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.57
Problem formulation.
Derive Chebyshev differentiation formulas using barycentric interpolation weights.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.58
Problem formulation.
Design a numerical experiment comparing second-order FDM, fourth-order FDM, and Chebyshev differentiation for f(x)=e^x on [-1,1].
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Add the Taylor expansions: f(x+h)+f(x-h) = 2f(x)+h^2f''(x)+\frac{h^4}{12}f^{(4)}(x)+O(h^6). Therefore f''(x)= \frac{f(x-h)-2f(x)+f(x+h)}{h^2}+O(h^2).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.59
Problem formulation.
Relate the rate of Chebyshev convergence to the distance of complex singularities from [-1,1].
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.60
Problem formulation.
Explain why equispaced global polynomial differentiation can be unstable and how Chebyshev nodes help.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.61
Problem formulation.
Investigate how norms and spectral properties of Chebyshev differentiation matrices grow with N.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.62
Problem formulation.
Use Chebyshev differentiation to discretize -u''(x)=f(x),\qquad u(-1)=u(1)=0.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.63
Problem formulation.
Explain why Chebyshev endpoint clustering is useful for boundary layers.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.64
Problem formulation.
Discuss aliasing in spectral differentiation and its effect on nonlinear problems.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.65
Problem formulation.
Design a spectral filter for reducing noise in Chebyshev differentiation.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.66
Problem formulation.
Construct tensor-product differentiation matrices for a rectangular domain.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.67
Problem formulation.
Compare memory and computational cost of FDM differentiation matrices and Chebyshev differentiation matrices.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
Check both the singular kernel evaluation and the long-history summation; fractional errors often come from the memory term rather than the algebraic solver.
Exercise 8.68
Problem formulation.
Compare finite differences, complex-step differentiation, symbolic differentiation, and automatic differentiation.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.69
Problem formulation.
Analyze the error f'-p_N' when p_N is an interpolation polynomial.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.70
Problem formulation.
Explain the role of Lebesgue constants in global polynomial differentiation.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.71
Problem formulation.
Prove a rigorous Chebyshev derivative error bound for analytic functions in a Bernstein ellipse.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.72
Problem formulation.
Derive Chebyshev differentiation error estimates for functions of finite Sobolev regularity.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.73
Problem formulation.
Study Gegenbauer reconstruction for recovering spectral accuracy near discontinuities.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.74
Problem formulation.
Investigate spectral viscosity methods for stabilizing spectral differentiation in nonsmooth problems.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.75
Problem formulation.
Study summation-by-parts finite-difference operators and their role in stable PDE discretizations.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.76
Problem formulation.
Analyze compact finite-difference schemes using Padé approximations and modified wavenumbers.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.77
Problem formulation.
Develop a statistical error model for numerical differentiation from noisy data.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The order claim should be confirmed by a Taylor expansion or a log-log refinement table using a problem with a known exact solution.
Exercise 8.78
Problem formulation.
Study Gaussian-process regression as a method for differentiating noisy observations.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.79
Problem formulation.
Develop an L^1-regularized differentiation method for signals with sparse derivative structure.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.80
Problem formulation.
Compare classical numerical differentiation with fractional differentiation matrices for nonlocal operators.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
Check both the singular kernel evaluation and the long-history summation; fractional errors often come from the memory term rather than the algebraic solver.
Exercise 8.81
Problem formulation.
Use Chebyshev differentiation matrices to discretize a nonlinear boundary-value problem and analyze Newton linearization.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
Verify the stationarity residual, feasibility residual, and descent or curvature condition, not only the final numerical value.
Exercise 8.82
Problem formulation.
Analyze severe time-step restrictions caused by Chebyshev differentiation matrices in explicit time stepping.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.83
Problem formulation.
Develop a matrix-free Chebyshev spectral differentiation algorithm using transforms.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.84
Problem formulation.
Study ultraspherical spectral methods and compare their conditioning with classical Chebyshev collocation.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
Distinguish forward error from backward error; a small residual does not imply a small forward error when the problem is ill-conditioned.
Exercise 8.85
Problem formulation.
Design an adaptive numerical differentiation method that chooses both stencil width and step size.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.86
Problem formulation.
Develop interval-arithmetic certificates for finite-difference derivative approximations.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.87
Problem formulation.
Design reproducible experiments comparing FDM and Chebyshev differentiation across smooth, analytic, nonsmooth, and noisy functions.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.88
Problem formulation.
Use tensor-product Chebyshev differentiation matrices to discretize a two-dimensional elliptic PDE and analyze the resulting linear system.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.89
Problem formulation.
Design a hybrid method using finite differences in one coordinate and Chebyshev differentiation in another.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Exercise 8.90
Problem formulation.
Compare finite differences, automatic differentiation, and spectral differentiation for physics-informed neural networks.
Method.
Use Taylor expansions, interpolation-based finite differences, Richardson extrapolation, roundoff-truncation balance, and differentiation matrices.
Detailed solution and justification.
Finite-difference formulas are obtained by matching Taylor expansions. For nodes x+jh, choose coefficients c_j so that \sum_j c_j f(x+jh) matches the desired derivative and cancels unwanted lower-order terms. The order is the first nonzero term left in the Taylor expansion. For spectral differentiation, construct the interpolation polynomial p_N through the nodes and set f'(x_i)\approx p_N'(x_i)=\sum_j D_{ij}f(x_j).
Conclusion.
The displayed derivation gives the requested formula, proof, or computation. The final expression should be checked against the assumptions and notation of the corresponding chapter.
Diagnostic comment.
The formula should be verified by substitution into the defining equation and, when possible, by a small numerical test.
Interactive tools
The theory and solutions come first. These tools are for review, laboratories, games, memory cards, and randomized assessment.