Numerical Methods Notes FE Sem 1 Applied Maths-I | Mumbai Univ

Most equations met in engineering, such as x log x = 1.2 or x e^x = cos x, cannot be solved exactly, and large systems of linear equations are tedious to solve by hand. This module teaches iterative numerical methods that give answers to any desired accuracy: the Newton-Raphson and Regula Falsi methods for a single equation, and the Gauss-Jacobi and Gauss-Seidel methods for systems of linear equations. These methods are the basis of the SCILAB programs written in the tutorials.

Summary

Transcendental and algebraic equations (6.1): an equation f(x) = 0 is algebraic if f is a polynomial and transcendental if it contains exponential, logarithmic or trigonometric terms. A root is first located by the intermediate value property: if f is continuous and f(a) and f(b) have opposite signs, a root lies between a and b. Numerical methods then refine an approximation step by step.

Newton-Raphson method (6.1): starting from an approximation \( x_0 \) near the root, the tangent at \( (x_0, f(x_0)) \) meets the x-axis at \( x_1 = x_0 - \frac{f(x_0)}{f'(x_0)} \). Repeating gives \( x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)} \). It follows from Taylor's series: \( f(x_0 + h) \approx f(x_0) + hf'(x_0) = 0 \Rightarrow h = -\frac{f(x_0)}{f'(x_0)} \).

Features of Newton-Raphson: convergence is quadratic (the number of correct digits roughly doubles each step) when \( x_0 \) is close to the root, but the method fails or misbehaves if \( f'(x_n) \) is zero or very small, and it needs the derivative. A special case gives square roots: \( x_{n+1} = \frac12\left(x_n + \frac{N}{x_n}\right) \) for \( \sqrt N \).

Regula Falsi (false position) method (6.1): choose a and b with f(a) f(b) < 0, join (a, f(a)) and (b, f(b)) by a chord, and take its x-intercept \( x_1 = \frac{af(b) - bf(a)}{f(b) - f(a)} \) as the next approximation. Replace whichever end point has f of the same sign as \( f(x_1) \), so the root always stays bracketed, and repeat. The method always converges for a continuous function but usually more slowly (linearly) than Newton-Raphson, and it needs no derivative.

Iterative methods for linear systems (6.2): for the system \( a_1x + b_1y + c_1z = d_1 \), \( a_2x + b_2y + c_2z = d_2 \), \( a_3x + b_3y + c_3z = d_3 \), first arrange the equations so that the system is diagonally dominant, \( |a_1| \ge |b_1| + |c_1| \), \( |b_2| \ge |a_2| + |c_2| \), \( |c_3| \ge |a_3| + |b_3| \) (strict in at least one row). Diagonal dominance is a sufficient condition for convergence. Then solve each equation for its diagonal unknown: \( x = \frac{1}{a_1}(d_1 - b_1y - c_1z) \), \( y = \frac{1}{b_2}(d_2 - a_2x - c_2z) \), \( z = \frac{1}{c_3}(d_3 - a_3x - b_3y) \).

Gauss-Jacobi method (6.2): start with an initial guess (usually x = y = z = 0) and compute all new values using only the values from the previous iteration, i.e. \( x^{(k+1)} = \frac{1}{a_1}(d_1 - b_1y^{(k)} - c_1z^{(k)}) \) and similarly for y and z. Continue until two successive iterations agree to the required accuracy.

Gauss-Seidel method (6.2): the same rearranged equations are used, but each newly computed value is used immediately: \( x^{(k+1)} \) from \( y^{(k)}, z^{(k)} \); \( y^{(k+1)} \) from \( x^{(k+1)}, z^{(k)} \); \( z^{(k+1)} \) from \( x^{(k+1)}, y^{(k+1)} \). Because the latest values are used, Gauss-Seidel generally converges faster than Gauss-Jacobi, roughly twice as fast for many diagonally dominant systems.

Self-learning topics: indeterminate forms such as 0/0 and ∞/∞ evaluated by L'Hospital's rule, \( \lim_{x\to a}\frac{f(x)}{g(x)} = \lim_{x\to a}\frac{f'(x)}{g'(x)} \) when the original limit is of that form; and the direct (non-iterative) Gauss elimination and Gauss-Jordan methods for linear systems, which reduce the augmented matrix to upper triangular and diagonal form respectively. All these methods are implemented as SCILAB programs in the term-work tutorials.

Mind map

Algorithm: Newton-Raphson method

Important definitions

Formulas

Key points to remember

• Always show the sign change f(a) f(b) < 0 to locate the root before iterating.
• Use radians, not degrees, for trigonometric functions in transcendental equations.
• Newton-Raphson converges fast near the root but can fail when f'(x) is close to zero or x₀ is far away.
• Regula Falsi always keeps the root between two points of opposite sign; one end point often stays fixed.
• Rearrange a linear system to be diagonally dominant before applying Jacobi or Seidel iteration.
• In Gauss-Jacobi use only old values; in Gauss-Seidel use each new value as soon as it is found.
• Gauss-Seidel generally needs fewer iterations than Gauss-Jacobi for the same accuracy.
• Carry at least one more decimal place in working than required in the answer.

Solved examples

Example 1. Find a real root of \( x^3 - 2x - 5 = 0 \) by the Newton-Raphson method, correct to four decimal places.

f(x) = x³ − 2x − 5, f'(x) = 3x² − 2. f(2) = −1, f(3) = 16, so a root lies between 2 and 3, close to 2. Take x₀ = 2.

x₁ = 2 − f(2)/f'(2) = 2 − (−1)/10 = 2.1.

f(2.1) = 9.261 − 4.2 − 5 = 0.061, f'(2.1) = 13.23 − 2 = 11.23; x₂ = 2.1 − 0.061/11.23 = 2.094568.

f(2.094568) ≈ 0.000186, f'(2.094568) ≈ 11.161647; x₃ = 2.094568 − 0.000186/11.161647 ≈ 2.094551.

x₂ and x₃ agree to four decimals, so the root is 2.0946.

Example 2. Using the Newton-Raphson method, find \( \sqrt{12} \) correct to four decimal places.

Let f(x) = x² − 12; then \( x_{n+1} = x_n - \frac{x_n^2 - 12}{2x_n} = \frac12\left(x_n + \frac{12}{x_n}\right) \).

Since 3.5² = 12.25, take x₀ = 3.5.

x₁ = ½(3.5 + 12/3.5) = ½(3.5 + 3.428571) = 3.464286.

x₂ = ½(3.464286 + 12/3.464286) = ½(3.464286 + 3.463918) = 3.464102.

x₃ = 3.464102 (no change). Hence √12 = 3.4641.

Example 3. Find the root of \( x\log_{10}x = 1.2 \) by the Newton-Raphson method, correct to three decimal places.

f(x) = x log₁₀x − 1.2, f'(x) = log₁₀x + 0.4343.

f(2) = 0.6021 − 1.2 = −0.5979, f(3) = 1.4314 − 1.2 = 0.2314: root between 2 and 3. Take x₀ = 2.5.

f(2.5) = 2.5(0.39794) − 1.2 = −0.20515, f'(2.5) = 0.39794 + 0.43429 = 0.83224; x₁ = 2.5 + 0.20515/0.83224 = 2.746505.

f(2.746505) ≈ 0.005113, f'(2.746505) ≈ 0.873075; x₂ = 2.746505 − 0.005856 = 2.740649.

f(2.740649) ≈ 0.000003, so x₃ ≈ 2.740646.

Root = 2.741 (to three decimal places).

Example 4. Find a real root of \( x^3 - 2x - 5 = 0 \) by the Regula Falsi method (four iterations).

f(2) = −1, f(3) = 16, so a = 2, b = 3.

Iteration 1: \( x_1 = \frac{2(16) - 3(-1)}{16 - (-1)} = \frac{35}{17} = 2.05882 \); f(x₁) = −0.39080 < 0, so the root lies in (2.05882, 3).

Iteration 2: \( x_2 = \frac{2.05882(16) - 3(-0.39080)}{16 + 0.39080} = 2.08126 \); f(x₂) = −0.14720 < 0, root in (2.08126, 3).

Iteration 3: x₃ = 2.08964; f(x₃) = −0.05468 < 0, root in (2.08964, 3).

Iteration 4: x₄ = 2.09274; f(x₄) = −0.02020 < 0.

The approximations increase steadily towards the root (≈ 2.0946 after further iterations), showing the slower, one-sided convergence of Regula Falsi compared with Newton-Raphson.

Example 5. Find the root of \( x^3 - x - 1 = 0 \) lying between 1 and 2 by the Regula Falsi method (three iterations).

f(1) = −1, f(2) = 5.

\( x_1 = \frac{1(5) - 2(-1)}{5 - (-1)} = \frac{7}{6} = 1.16667 \); f(1.16667) = −0.57870, root in (1.16667, 2).

\( x_2 = \frac{1.16667(5) - 2(-0.57870)}{5 + 0.57870} = 1.25311 \); f(1.25311) = −0.28536, root in (1.25311, 2).

\( x_3 = \frac{1.25311(5) - 2(-0.28536)}{5 + 0.28536} = 1.29344 \); f(1.29344) = −0.12954.

After three iterations the root is approximately 1.2934; continuing the process converges to 1.3247.

Example 6. Solve by the Gauss-Jacobi method: 10x + y + z = 12, x + 10y + z = 12, x + y + 10z = 12.

The system is diagonally dominant (10 > 1 + 1 in each row).

x = (12 − y − z)/10, y = (12 − x − z)/10, z = (12 − x − y)/10. Start with x = y = z = 0.

Iteration 1: x = y = z = 12/10 = 1.2.

Iteration 2: x = y = z = (12 − 2.4)/10 = 0.96.

Iteration 3: x = y = z = (12 − 1.92)/10 = 1.008.

Iteration 4: x = y = z = (12 − 2.016)/10 = 0.9984.

The values approach x = y = z = 1, which is the exact solution.

Example 7. Solve by the Gauss-Seidel method: 20x + y − 2z = 17, 3x + 20y − z = −18, 2x − 3y + 20z = 25.

The system is diagonally dominant. x = (17 − y + 2z)/20, y = (−18 − 3x + z)/20, z = (25 − 2x + 3y)/20. Start with y = z = 0.

Iteration 1: x = 17/20 = 0.8500; y = (−18 − 2.55 + 0)/20 = −1.0275; z = (25 − 1.7 − 3.0825)/20 = 1.0109.

Iteration 2: x = (17 + 1.0275 + 2.0218)/20 = 1.0025; y = (−18 − 3.0075 + 1.0109)/20 = −0.9998; z = (25 − 2.0050 − 2.9994)/20 = 0.9998.

Iteration 3: x = 1.0000, y = −1.0000, z = 1.0000.

Solution: x = 1, y = −1, z = 1.

Example 8. Solve 27x + 6y − z = 85, 6x + 15y + 2z = 72, x + y + 54z = 110 by the Gauss-Seidel method, correct to three decimal places.

Diagonally dominant. x = (85 − 6y + z)/27, y = (72 − 6x − 2z)/15, z = (110 − x − y)/54. Start with y = z = 0.

Iteration 1: x = 3.1481, y = (72 − 18.8889)/15 = 3.5407, z = (110 − 3.1481 − 3.5407)/54 = 1.9132.

Iteration 2: x = 2.4322, y = 3.5720, z = 1.9258.

Iteration 3: x = 2.4257, y = 3.5729, z = 1.9260.

Iteration 4: x = 2.4255, y = 3.5730, z = 1.9260.

Solution: x = 2.426, y = 3.573, z = 1.926.

Important questions and answers

Q1. Derive the Newton-Raphson formula. (3 marks)

Let x₀ be an approximation to a root of f(x) = 0 and let x₀ + h be the exact root, so f(x₀ + h) = 0.

By Taylor's series, \( f(x_0) + hf'(x_0) + \frac{h^2}{2!}f''(x_0) + \cdots = 0 \). Neglecting h² and higher powers (h small), \( h = -\frac{f(x_0)}{f'(x_0)} \).

So a better approximation is \( x_1 = x_0 - \frac{f(x_0)}{f'(x_0)} \), and in general \( x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)} \).

Geometrically, x₁ is where the tangent at (x₀, f(x₀)) cuts the x-axis.

Q2. Find the root of \( x - \cos x = 0 \) by the Newton-Raphson method, correct to three decimal places. (4 marks)

f(x) = x − cos x, f'(x) = 1 + sin x (x in radians). f(0) = −1, f(1) = 0.4597: root in (0, 1). Take x₀ = 0.5.

x₁ = 0.5 − (−0.377583)/1.479426 = 0.755222.

x₂ = 0.755222 − 0.027103/1.685451 = 0.739142.

x₃ = 0.739142 − 0.000095/1.673654 = 0.739085.

Root = 0.739.

Q3. Find a root of \( xe^x = \cos x \) by the Regula Falsi method (three iterations). (5 marks)

f(x) = xe^x − cos x. f(0) = −1, f(1) = 2.71828 − 0.54030 = 2.17798: root in (0, 1).

\( x_1 = \frac{0(2.17798) - 1(-1)}{2.17798 + 1} = 0.31467 \); f(0.31467) = −0.51987 < 0, root in (0.31467, 1).

\( x_2 = \frac{0.31467(2.17798) - 1(-0.51987)}{2.17798 + 0.51987} = 0.44673 \); f(0.44673) = −0.20354 < 0, root in (0.44673, 1).

\( x_3 = \frac{0.44673(2.17798) - 1(-0.20354)}{2.17798 + 0.20354} = 0.49402 \); f(0.49402) = −0.07080.

Approximate root after three iterations 0.494; further iterations converge to 0.5178.

Q4. Write the Regula Falsi formula and explain the method graphically. (3 marks)

If f(a) and f(b) have opposite signs, the chord joining A(a, f(a)) and B(b, f(b)) cuts the x-axis at \( x_1 = \frac{af(b) - bf(a)}{f(b) - f(a)} \).

x₁ is taken as the next approximation. If f(a) f(x₁) < 0 the root lies in (a, x₁), otherwise in (x₁, b); the process is repeated with the new interval.

Since the curve is replaced by a straight chord (a 'false position'), the method needs no derivative and always keeps the root bracketed.

Q5. Compare the Gauss-Jacobi and Gauss-Seidel methods. (3 marks)

Both rearrange a diagonally dominant system to express each unknown in terms of the others and iterate from an initial guess.

Gauss-Jacobi uses values from the previous iteration only (simultaneous displacement); Gauss-Seidel uses each new value immediately (successive displacement).

Gauss-Seidel usually converges faster and needs less storage; Gauss-Jacobi is easier to parallelise.

Q6. Solve 27x + 6y − z = 85, 6x + 15y + 2z = 72, x + y + 54z = 110 by the Gauss-Jacobi method (five iterations). (6 marks)

x = (85 − 6y + z)/27, y = (72 − 6x − 2z)/15, z = (110 − x − y)/54, starting from (0, 0, 0).

Iteration 1: (3.1481, 4.8000, 2.0370).

Iteration 2: (2.1569, 3.2691, 1.8898).

Iteration 3: (2.4917, 3.6852, 1.9366).

Iteration 4: (2.4009, 3.5451, 1.9226).

Iteration 5: (2.4316, 3.5833, 1.9269).

The values oscillate towards x = 2.426, y = 3.573, z = 1.926, but more slowly than Gauss-Seidel.

Q7. Solve by Gauss-Jacobi: 20x + y − 2z = 17, 3x + 20y − z = −18, 2x − 3y + 20z = 25 (four iterations). (5 marks)

x = (17 − y + 2z)/20, y = (−18 − 3x + z)/20, z = (25 − 2x + 3y)/20, starting from (0, 0, 0).

Iteration 1: x = 0.85, y = −0.90, z = 1.25.

Iteration 2: x = (17 + 0.9 + 2.5)/20 = 1.02, y = (−18 − 2.55 + 1.25)/20 = −0.965, z = (25 − 1.7 − 2.7)/20 = 1.03.

Iteration 3: x = 1.0012, y = −1.0015, z = 1.0032.

Iteration 4: x = 1.0004, y = −1.0000, z = 0.9997.

Solution x = 1, y = −1, z = 1.

Q8. Why must the equations be diagonally dominant for iterative methods? Rearrange x + y + 54z = 110, 27x + 6y − z = 85, 6x + 15y + 2z = 72 suitably. (3 marks)

Diagonal dominance is a sufficient condition for the Jacobi and Seidel iterations to converge; without it the iterates may diverge.

Rearranged: 27x + 6y − z = 85 (27 > 7), 6x + 15y + 2z = 72 (15 > 8), x + y + 54z = 110 (54 > 2).

Q9. Evaluate \( \lim_{x\to0}\frac{e^x - 1 - x}{x^2} \) using L'Hospital's rule (self-learning). (3 marks)

Form 0/0. Differentiate numerator and denominator: \( \lim_{x\to0}\frac{e^x - 1}{2x} \), still 0/0.

Again: \( \lim_{x\to0}\frac{e^x}{2} = \frac12 \).

Exam tips

• Show the sign change of f(x) to justify the interval or starting value; it carries a mark.
• Tabulate iterations (n, x_n, f(x_n), f'(x_n), x_{n+1}) neatly; it is faster to check and easier to mark.
• Set your calculator to radian mode for trigonometric equations.
• Stop only when two successive values agree to the required number of decimals, and state the final answer clearly.
• Check diagonal dominance and rearrange the equations before starting Jacobi or Seidel; write the rearranged formulas.
• In Gauss-Seidel, clearly show that the new x is used in the y-equation and the new x, y in the z-equation.

FAQs

Which is better, Newton-Raphson or Regula Falsi?

Newton-Raphson converges much faster (quadratically) when started near the root, but needs the derivative and can fail if f'(x) is near zero. Regula Falsi is slower but safer because it always keeps the root bracketed.

What if the system is not diagonally dominant?

First try reordering the equations. If no ordering is diagonally dominant, the iterations may still converge but convergence is not guaranteed; a direct method such as Gauss elimination is safer.

How many iterations should I show in the exam?

Continue until two successive iterations agree to the accuracy asked. If the question specifies the number of iterations, show exactly that many.

The module title mentions expansion of function; what is examined?

The detailed contents of the NEP syllabus list Newton-Raphson, Regula Falsi, Gauss-Jacobi and Gauss-Seidel methods, with indeterminate forms (L'Hospital's rule), Gauss elimination and Gauss-Jordan as self-learning topics. Prepare these first.

These notes follow the University of Mumbai first-year engineering syllabus (NEP 2020, Applied Mathematics-I) and were prepared with AI help and checked.