Numerical differentiation. Calculation of the first derivative.

Definition

By definition, the first derivative of a smooth function \(f(x)\) at a point x is calculated as

\begin{equation} f^{\prime}(x)=\lim _{h \rightarrow 0} \frac{f(x+h)-f(x)}{h}. \end{equation}

When calculating the first derivative of the function \(f(x)\) on a computer, we replace the infinitesimal \(h \rightarrow \infty\) with a small but finite value \(h\):

\begin{equation} f^{\prime}(x)=\frac{f(x+h)-f(x)}{h}+\mathrm{O}(h), \end{equation}

where \(\mathrm{O}(h)\) is the derivative calculation error, which naturally depends on \(h\). Previous formula is called a difference scheme for calculating the first derivative (more precisely, a right difference scheme or just a right difference). Similarly, maybe the left-hand difference scheme is written.

How to determine \(\mathrm{O}(h)\)? Expand the function \(f(x)\) in a Taylor series at the point \(x + h\):

\begin{equation} f(x+h)=f(x)+h f^{\prime}(x)+\frac{h^{2}}{2} f^{\prime \prime}(x)+\frac{h^{3}}{6} f^{\prime \prime \prime}(x)+\ldots, \end{equation}

whence it follows that in the first order of the expansion

\begin{equation} \mathrm{O}(h)=-\frac{h}{2} f^{\prime \prime}(x)+\ldots. \end{equation}

By choosing a very small \(h\), the round-off errors in computing on a computer can be comparable to or greater than \(h\). Therefore, we are interested in an algorithm that gives lower error value for the same value of \(h\).

Such an improved algorithm can be easily obtained by expanding the function \(f(x)\) into a Taylor series at the points \(x + h\) and \(x - h\), then subtracting one result from the other, which gives

\begin{equation} f^{\prime}(x)=\frac{f(x+h)-f(x-h)}{2 h}+\mathrm{O}\left(h^{2}\right), \end{equation}

where the error in calculating the first derivative

\begin{align*} \mathrm{O}\left(h^{2}\right)=-\frac{h^{2}}{6} f^{\prime \prime \prime}(x)+\ldots. \end{align*}

This is the central difference scheme (central difference).

In principle, it is possible to follow the path of improving the accuracy of the method for calculating the first derivative and further. For example, considering the expansion of the function \(f(x)\) in a Taylor series at the points \(x + h\), \(x + 2h\), \(x - h\), and \(x - 2h\), one can obtain a four-point scheme etc.

Usage

Imagine that we want to find the derivative of the following function:

\begin{equation} f(x) = \sin{(x)} \end{equation}

Then the code will look like this:

// example_first_order_derivative_h.cpp

#include <iostream>
#include "../src/numerary.hpp" // Numerary library

using namespace std;
using namespace numerary;

/* Functiion to derive */
double f(double x) {
    return sin(x);
}

/* The main function */
int main() {

    const short int order = 1;
    double x, dy_dx;

    // Point where we want get value of derivative function
    x = M_PI;

    dy_dx = Numerary::differentiate(f, order, x);

    cout << "dy/dx (" << x << ") = " << dy_dx << endl;

    return 0;
}