Newton’s Method

The Newton-Raphson method, usually shortened to Newton’s method, is a method of approximation that allows engineers to solve optimization problems. This method is used in many engineering problems including finding an equilibrium point and finding optimum points in a relationship or process.

Engineering Context

MAE: Mechanical and aerospace engineers use Newton’s Method to find the angle between the incoming wind and an aircraft that will place the airplane in an equilibrium state. The angle at which these two forces interact will have an effect on the direction of the aircraft’s movement. This angle, alpha, is referred to as the angle of attack and can be seen in Fig. 1. Using Newton’s Method, the angle of attack that places the aircraft in equilibrium can be found by setting the lift on the airplane, L, equal to the weight of the aircraft, W. If we call this relationship \( F = L − W \), then Newton’s method gives

\[ \alpha_{i+1} = \alpha_i - \frac{dF}{d\alpha} F_i \]

which can then be iterated until F ≈ 0 and the angle of attack stops changing.

Figure 1: An aircraft at a given angle of attack.

Figure 1: An aircraft at a given angle of attack.

ECE: Electrical engineers may need to create a controller that will stabilize a simple system that has a single control input. A common system is an inverted pendulum, shown in Fig. 2, which is governed by the equation of motion

\[ mL^2 \ddot{\theta}(t) + mgL \sin(\theta(t)) = u(t) - b\dot{\theta}(t) \]

where \( \theta \) is the angle the pendulum makes with the vertical, u is the control input, and m, L, g, and b are constants. We can use Newton’s method to find the control input u(t) that will keep the pendulum in the upright position (\( \theta = 0 \)). Rearranging the equation to pull everything to one side, we have

\[ f(u) = mL^2 \ddot{\theta}(t) + mgL \sin(\theta(t)) - u(t) + b\dot{\theta}(t) \]

which means that we are trying to find the u(t) that satisfies the equation of motion. Newton’s method is then used to find the appropriate u(t) by

\[ u_{i+1}(t) = u_i(t) - \frac{df(u)}{du} f(u_i) \]

This would require us to know what the derivatives of the angle are, but those could be estimated using data from previous time-steps.

Figure 2: An inverted pendulum.

Figure 2: An inverted pendulum.

BE: In order for cells to function, they perform a series of chemical reactions that involve breaking down molecules in order to create something new. These processes are called metabolic pathways, and they are absolutely essential to living organisms. Metabolic pathways are usually very complex, but we can look at a simplified version here.

The rate of a single enzyme-catalyzed reaction can be given by a Michaelis-Menten equation

\[ v = \frac{v_{\text{max}} S}{K_m + S} \]

where v is the rate of the reaction, S is the concentration of the substrate (the molecule being converted by the enzyme), and Km and vmax are constants.

Biological engineers could use Newton’s method to find the substrate concentration that gives a given reaction rate. If we want to find the reaction rate v0, we can define

\[ f(S) = \frac{v_{\text{max}} S}{K_m + S} - v_0 \]

Newton’s method can then be used to find the S that yields v0 using

\[ S_{i+1} = S_i - \frac{df(S)}{dS} f(S_i) \]

and iterating until \( S_{i+1} \approx S_i \).

CEE: The mass balance for a pollutant in a well-mixed lake can be written as

\[ V\frac{dV}{dt} = W - Qc - kV\sqrt{c} \]

where c is the concentration of the pollutant and the other variables are constants. The steady-state concentration of the pollutant is achieved when dc/dt = 0 and can be found using Newton’s method by iterating the relation

\[ c_{ss}^{(n+1)} = c_{ss}^{(n)} - \frac{ W - Qc_{ss}^{(n)} - kV \sqrt{ c_{ss}^{(n)} }}{ -Q - \frac{kV}{2\sqrt{c_{ss}^{(n)}}} } \]

until \( c_{ss}^{(n+1)} \approx c_{ss}^{(n)} \)

The Essentials

To implement Newton’s method, do the following:

  1. Find the derivative of the function with respect to the variable x, \( f′(x) \)
  2. Determine an initial guess \( x_0 \)
    1. If you want, you can make the algorithm converge faster by plugging in a few values of x into \( f(x) \) until you find two values \( x_1 \) and \( x_2 \) that cause \( f(x) \) to change signs
  3. Starting with x0, complete the first iteration using the algorithm
    \[ x_1 = x_0 - \frac{f(x_0)}{f'(x_0)} \]
  4. Continue iterating until \( x_{i+1} \approx x_i \)
  5. Check that \( f(x_{i+1}) \approx 0 \)

Example

The following example shows how to use Newton’s method to calculate where \( f(x) = 0 \) for the given equation.

\[ f(x) = x^5 + 2x^3 - 1 \]

1. We start by trying a few values to find a point where the sign of y changes

\[ f(0) = 0^5 + 2(0)^3 - 1 = -1 \]
\[ f(1) = 1^5 - + 2(1)^3 - 1 = 2 \]

We found an x value that gives us a negative y and one that gives a positive y! Now we can move to step 2.

2. Since we know that our x value will be between 0 and 1, lets choose .5 to be our \( x_0 \) value.

3. Plug \( x_0 \) into the formula above

\[ x_1 = 0.5 - \frac{(0.5)^5 + 2(0.5)^3 - 1}{5(0.5)^4 + 6(0.5)^2} = 0.5 - \frac{-0.7188}{1.8125} \approx 0.8966 \]

4. Now we do that again with our new \( x_1 \) value

\[ x_2 = 0.8966 - \frac{(0.8966)^5 + 2(0.8966)^3 - 1}{5(0.8966)^4 + 6(0.8966)^2} = 0.8966 - \frac{1.0209}{8.0546} \approx 0.7698 \]
\[ x_3 = 0.7698 - \frac{(0.7698)^5 + 2(0.7698)^3 - 1}{5(0.7698)^4 + 6(0.7698)^2} = 0.7698 - \frac{0.1827}{5.3114} \approx 0.7354 \]
\[ x_4 = 0.7354 - \frac{(0.7354)^5 + 2(0.7354)^3 - 1}{5(0.7354)^4 + 6(0.7354)^2} = 0.7354 - \frac{0.0105}{4.707} \approx 0.733 \]

At this point, we can plug this x value back into \( f(x) \) to see if we are close to \( f(x) = 0 \).

\[ f(0.733) = -2.654 \times 10^{-4} = -0.0002654 \]

The more we repeat the Newton’s Method Formula, the closer our approximation will be to the true x value.

A Deeper Dive

When we use Newton’s Method to approximate where \( f(x) = 0 \), we can think of it as essentially choosing a point on the function where we will draw a tangent line. This tangent line will intersect the x-axis at some point and that point will be used as our new x value, from which we will find a new tangent line. Ideally, the location where this new tangent line intersects the x-axis will be even closer to the actual \( f(x) = 0 \) value than our original tangent line! This idea can be expressed using the following equation.

\[ f'(x_0) = \frac{\text{rise}}{\text{run}} = \frac{f(x_0)}{x_0 - x_1} \]

By simplifying this equation we end up with the equation for Newton’s method shown in the essentials section.

\[ x_1 = x_0 - \frac{f(x_0)}{f'(x_0)} \]

Figure 3 shown below, gives a visual example of Newton’s Method. Here there is a tangent line drawn where \( x = 10 \), representing our initial guess \( x_0 \). This line intersects the x-axis at \( x = 5 \), which gives us an \( x_1 = 5 \). When we draw a tangent line at our new point \( f(5) \), notice that where our tangent line for \( f(5) \) crosses the x-axis is closer to the actual point, \( f(x) = 0 \), than where our tangent line for \( f(10) \) crosses the x-axis.

Figure 3 -a graph example of Newton's Method

Figure 3: \( y = \frac{1}{8x^2} \) with a tangent line drawn at \( x=10 \) and \( x=5 \)

Practice

Use Newton’s Method to find the x value that makes \( f(x) = 0 \) for the following function.

\[ y = x^4 - 2x^2 - 1 \]

Solution:

Using the same method as is shown in the example section, the answer should be roughly \( x = 1.554 \)

More Resources