Runge-Kutta Methods

We have seen that the Taylor series method has desirable features, particularly in its ability to keep the errors small, but that it also has the strong disadvantage of requiring the evaluation of higher derivatives of the function f(x,y). In the Taylor series method, each of these higher order derivatives is evaluated at the point x_i at the beginning of the step, in order to evaluate y(x_i+1) at the end of the step. We observed that the Euler method could be improved by computing the function f(x,y) at a predicted point at the far end of the step in x. The Runge-Kutta approach is to aim for the desirable features of the Taylor series method, but with the replacement of the requirement for the evaluation of higher order derivatives with the requirement to evaluate f(x,y) at some points within the step x_i to x_i+1. Since it is not initially known at which points in the interval these evaluations should be done, it is possible to choose these points in such a way that the result is consistent with the Taylor series solution to some particular, which we shall call the order of the Runge-Kutta method.

We begin with a simple case, since the derivation is straightforward, and while this case is not particularly useful in practice, it is useful for understanding the Runge-Kutta methods. We begin by writing the Taylor series for the solution y(x) in the form:

$$y(x+h) = y(x) + h f(x,y) + \frac{h^2}{2}y^{\prime\prime}(x) + {\cal O}(h^3)\,.$$

We have used the differential equation $y^{\prime}(x) = f(x,y)$ to evaluate $y{\prime}(x)$, but it remains to evaluate $y^{\prime\prime}(x)$. Using the earlier result for the differentiation of a function like f(x,y(x)), we have:

$$y^{\prime\prime}(x) = f_x(x,y) + f_y(x,y) y^{\prime}(x)\,,$$

where the subscript x indicates partial differentiation with respect to x, holding y constant, and similarly for the subscript y. Since $y^{\prime}(x) = f(x,y)$, we can write this as:

$$y^{\prime\prime}(x) = f_x(x,y) + f_y(x,y) f(x,y)\,,$$

and the Taylor series becomes:

$$y(x+h) = y(x) + h f(x,y) + \frac{h^2}{2} (f_x(x,y) + f_y(x,y) f(x,y)) + {\cal O}(h^3)\,.$$

The Runge-Kutta method assumes that the correct value of the slope over the step can be written as a linear combination of the function f(x,y) evaluated at certain points in the step. In the method of order 2 this results in writing the iteration step in the form:

$$y(x+h) = y(x) + A h f_0 + B h f_1\,,$$

where:

$$f_0 = f(x,y) \,\,\,\,{\rm and} \,\,\,\, f_1 = f(x + P h,y + Q h f_0)\,.$$

The constants A, B, P, and Q are still to be determined, which we shall do by comparing the Runge-Kutta formula with the second order Taylor series given above. While f₀ involves only information already available at the initial position (x,y), f₁ must be evaluated by developing the Taylor expansion of f(x,y) about the point x. Since f(x,y) is a function of both x and y, the first order Taylor series for f(x,y) can be written as:

$$f_1 = f(x,y) + f_x(x,y) P h + f_y(x,y) Q h f_0 + {\cal O}(h^2)\,.$$

This can then be used in the Runge-Kutta formula for y(x+h) to obtain:

$$y(x+h) = y(x) + (A + B) h f(x,y) + B h^2 P f_x(x,y) + B h^2 Q f_y(x,y) f(x,y) + {\cal O}(h^3)\,.$$

On comparing this with the direct Taylor series for y(x), we have:

$$A + B = 1 \,\,\,\,\, BP = \frac{1}{2} \,\,\,\,\, BQ = \frac{1}{2}\,.$$

We thus have three conditions on the four constants such that the direct Taylor series and the Runge-Kutta formula will agree to second order in h.

There are a number of interesting choices we can make, since we have only three conditions on the four constants A, B, P, and Q. If we choose A = 1/2, we have B = 1/2 and P = Q = 1, which leads to the Runge-Kutta formula:

$$y(x+h) = y(x) + \frac{h}{2}(f(x,y) + f(x+h,y+h f(x,y)))\,,$$

which is just the Heun method encountered earlier.

The choice of A = 0 leads to B = 1 and $P = Q = \frac{1}{2}$, so that the Runge-Kutta formula takes the form:

$$y(x+h) = y(x) + h f\left(x + \frac{h}{2},y + \frac{h}{2} f(x,y)\right)\,,$$

which is a new method related to the Euler method, sometimes called the midpoint method, for obvious reasons.