ง่าย ๆ ก็คือให้ มากับ Initial condition (ค่าเริ่มต้น) แล้วหาว่า ธรรมดา คือไรเอ่ย?
16 - Boundary Value Problems for Ordinary Differential Equations (BVP)
Initial Value Problems for First-Order Ordinary Differential Equations
Definition
- Consider the following problem:
- เรามีสมการเชิงอนุพันธ์ 1 สมการ (ฝั่งซ้ายคือ )
- และมี เงื่อนไขเริ่มต้น (initial condition) คือ
→ บอกว่า “ตอนเริ่มเวลา 0, ค่าของ y คืออะไร”
- We want to find the unknown function that satisfies the above equation
- เป้าหมาย: หาว่า (ฟังก์ชันของเวลา ) มีหน้าตาเป็นยังไง
Examples
- These are known as initial value problems (IVP) for first-order ordinary differential equations (ODE)
Think of an IVP like knowing your starting position and speed - you're trying to figure out your entire path from there.
Types of Differential Equations
Differential Equations
- Equations with derivatives of some unknown functions
สมการที่มี อนุพันธ์ (derivative) ของฟังก์ชันที่ไม่รู้จัก
Ordinary Differential Equations (ODE)
- Differential equations where all derivatives are ordinary derivatives
- Example:
Think of ODEs like tracking how something changes over time - temperature of coffee cooling, population growth, etc.
No PARTIAL DERIVATIVE
Partial Differential Equations (PDE)
- Differential equations where some derivatives are partial derivatives
- Example:
PDEs are like tracking how something changes across space AND time - like heat spreading across a metal plate.
มี SOME PARTIAL DERIVATIVE
First-Order ODE
สมการเชิงอนุพันธ์อันดับหนึ่ง (มีอนุพันธ์แค่ชั้นเดียว)
- ODE where all derivatives are first ordinary derivatives
- Example:
- อาจารย์บอกว่า mostly can always transform to standard form
Solving IVP Numerically
- Approximate by a sequence of discrete points:
where

บางสมการเชิงอนุพันธ์ แก้ด้วยสูตรหรืออินทิเกรตไม่ได้จริง ๆ
ดังนั้นเราจำเป็นต้อง “ประมาณคำตอบ” ด้วย ตัวเลข (numerical method) แทน
แทนที่เราจะหาฟังก์ชัน ที่ต่อเนื่องตลอดเวลา (วาดเป็นเส้นโค้งสวย ๆ)
เราจะ “แทน” มันด้วย จุด ๆ ที่คำนวณได้ทีละช่วงเวลา
"Stability" of an ODE
มันก็มี Properties ของ ODE อยู่ก็คือ Stability (แต่ไม่เหมือนกับที่เราเรียนก่อนหน้านี้นะ) ถ้าถูกต้องจริง ๆ ควรเรียกว่า Conditioning ด้วยซ้ำ
สมการนี้อ่อนไหวแค่ไหนต่อค่าตั้งต้นที่เปลี่ยนไปเล็กน้อย
Important Note
- Should be conditioning of an ODE, but traditionally (misleadingly) known as stability
Stable Solution
- A solution of the ODE is stable if:
- After perturbing the initial value (i.e., changing slightly)
- The perturbed solution remains close to the original solution
Asymptotically Stable Solution
- A stable solution is asymptotically stable if:
- Not only does the perturbed solution remain close to the original solution
- They converge toward each other over time
| ประเภท | ภาพในหัว | ตัวอย่างเปรียบเทียบ |
|---|---|---|
| Stable | เส้นคู่ขนานไปเรื่อย ๆ | รางรถไฟคู่ขนาน – ห่างเท่าเดิมเสมอ |
| Asymptotically Stable | เส้นที่ค่อย ๆ เข้าใกล้กัน | ทางสองสายที่ค่อย ๆ รวมเป็นเส้นเดียว |
| Unstable | เส้นที่ค่อย ๆ แยกห่างกัน | รถสองคันเริ่มใกล้กัน แต่ยิ่งขับยิ่งห่าง |
Example 1.1: Stability Analysis
Problem
Is the solution to
for a given real constant stable? If so, is it asymptotically stable?
Solution
- The solution to the IVP is:

- The solution is stable but NOT asymptotically stable
Like parallel roads - they never get closer or farther apart, just stay at the same distance forever.
เจอ (ค่าคงที่) → integrate ได้เลย
จากนั้นแทน initial condition หาค่า
คำตอบคือ
ซึ่ง stable แต่ไม่ asymptotically stable เพราะเส้นคู่ขนานกันตลอด
Example 1.2: Stability Analysis
รู้ว่า stable / unstable / asymptotically stable = รู้ว่า numerical method ไหนใช้ได้, step size h ควรเป็นเท่าไหร่, ใช้ explicit หรือ implicit ดี และรู้ว่าปัญหานั้น stiff ไหม
ex1.py
อาจารย์เอามาให้เรา “เห็นภาพ” ของความหมายคำว่า stable / unstable / asymptotically stable
Problem
Is the solution to
for a given real constant stable? If so, is it asymptotically stable?
Solution
Claim: The solution is
Verify:
and
Case 1: When

- The solution is UNSTABLE
Like compound interest - small differences in starting amounts grow exponentially over time.
Case 1:
โตขึ้นเรื่อย ๆ เมื่อ เพิ่ม
สมมติเริ่มจาก ต่างกันนิดเดียว → พอเวลาผ่านไป ความต่างจะ “ขยาย” อย่างรวดเร็ว
กราฟ: เส้นโค้งชี้ขึ้นทั้งหมด → แยกออกจากกัน
สรุป: 🔴 Unstable
Case 2: When

- The solution is ASYMPTOTICALLY STABLE
Like radioactive decay - all samples eventually converge toward zero, regardless of starting amount.
Case 2:
หดลงเรื่อย ๆ (เพราะ เป็นลบ → exponent ติดลบ → เข้าใกล้ 0)
ไม่ว่าค่าเริ่มต่างกันเท่าไร ทุกเส้นจะค่อย ๆ มุ่งเข้าหา
กราฟ: เส้นโค้งที่ค่อย ๆ ลดลงจนแบนที่ศูนย์
สรุป: 🟢 Asymptotically Stable
Euler's Method
อันนี้เป็น First Method ในการ Solve First-Order ODE
Derivation
Given:
Using Taylor series:
\begin{aligned} y(t+h) &= y(t) + y'(t)h + y''(t)\frac{h^2}{2} + \ldots\\ &= y(t) + f(t,y)h + y''(t)\frac{h^2}{2} + \ldots \end{aligned}$$ By **ignoring the second-order and higher terms** from the Taylor series, we get **Euler's Method (EM)**: >เอาแค่ 2 terms แรกมา $$\boxed{y_{k+1} = y_k + h_k f(t_k, y_k)}$$ where: - $y_k$ denotes our approximation of $y(t_k)$ - $h_k = t_{k+1} - t_k$ is called the **step size** - Step size เปลี่ยนได้นะ บางครั้งก็มีประโยชน์ด้วย อย่างถ้าไปดูกราฟข้างบ้าน ในช่วงแรก ๆ ที่มันเปลี่ยนเยอะ เราก็อาจจะทำให้ Step size มันถี่หน่อย to increase efficientcy - Use $(0, y_0)$ (given by the initial condition) as the first point > Like using just the slope at a point to predict the next point - simple but not super accurate. ### Example 2: Using Euler's Method #### Problem Use Euler's method on the IVP $$\frac{dy}{dt} = \frac{t}{y}; \quad y(0) = 1$$ with constant step size $h = 0.5$ #### Solution **Step 0:** - $(t_0, y_0) = (0, 1)$ **Step 1:** - $t_1 = t_0 + h = 0 + 0.5 = 0.5$ - $y_1 = y_0 + f(t_0, y_0)h_0 = y_0 + \left(\frac{t_0}{y_0}\right)h_0 = 1 + \left(\frac{0}{1}\right)(0.5) = 1$ - So $(t_1, y_1) = (0.5, 1)$ **Step 2:** - $t_2 = t_1 + h = 0.5 + 0.5 = 1$ - $y_2 = y_1 + f(t_1, y_1)h_1 = y_1 + \left(\frac{t_1}{y_1}\right)h_1 = 1 + \left(\frac{0.5}{1}\right)(0.5) = 1.25$ - So $(t_2, y_2) = (1, 1.25)$ **Step 3:** - $t_3 = t_2 + h = 1 + 0.5 = 1.5$ - $y_3 = y_2 + \left(\frac{t_2}{y_2}\right)h_2 = 1.25 + \left(\frac{1}{1.25}\right)(0.5) = 1.65$ - So $(t_3, y_3) = (1.5, 1.65)$ - And so on... #### Comparison to Analytical Solution >ไอ่ข้อข้างบนมันง่าย ใช้ Calculus ธรรมดามาหาสมการจริง ๆ เดี๋ยวก็ได้ออกมา แล้วเรามาเทียบค่ากันว่าเป็นยังไง The analytical solution is $y = \sqrt{t^2 + 1}$ This means: - $y(0.5) = 1.18$ - $y(1) = \sqrt{2} \approx 1.414$ - $y(1.5) = 1.8028$ #### Observations - As seen from the example, **EM is not quite accurate** - **Not recommended.** Too low order (explained later) ![[CSS322_Doable_L13_Ex2 1.pdf]] ### Terminology #### Numerical Integration - This process of approximating $y(t)$ is called **numerical integration** of the ODE - การที่ได้มาแต่ละจุด ๆ เนี่ยเรียกว่า numerical integration of the ODE #### One-Step Method - EM (Euler’s Method) is **one-step**: formula for $y_{k+1}$ involves only $y_k$ (not $y_{k-1}, y_{k-2}, \ldots$) - **==Use only the current point to get the next one.==** > Like taking each step based only on where you are now, not remembering previous steps. #### Explicit Method - EM is **explicit**: The equation is a formula for $y_{k+1}$, i.e., it can be turned into an assignment statement - คือถ้าดูจากสมการนี้ ${y_{k+1} = y_k + h_k f(t_k, y_k)}$ คือฝั่งขวา เรารู้ค่าหมด ดังนั้นเอาเข้าคอมคำนวณได้เลยล่ะ - ไม่เหมือน Implicit method (explain later) ที่ฝั่งขวาบางครั้งอาจจะมี $y_{k+1}$ > You can directly calculate the next value without solving any equations. --- ## Truncation Errors >Truncation Error คือมาจาก Method itself ไม่ใช่ในเรื่องของการ Rounding นะ — ที่เห็นได้ชัดเลยของ EM ก็คือเราไม่เอา Term หลัง ๆ จาก 2 terms แรกเลยไง5555 ### Two Types of Errors #### 1. Global Truncation Error - **Global truncation error** at the $k$-th step is the difference: $$e_k = y_k - y(t_k)$$ where: - $y_k$ is the computed solution at $t_k$ - $y(t)$ is the true solution of the IVP > How far off you are from the true path after accumulating all previous errors. >คือ “ความผิดพลาดสะสมหลังจากหลายก้าว” เพราะในความเป็นจริง ทุก step ก่อนหน้าก็มี error นิด ๆ อยู่แล้ว เวลาคำนวณ step ต่อไป error ก็สะสมและขยายเรื่อย ๆ #### 2. Local Truncation Error - Assume all the previous steps are exact, i.e., $y_i = y(t_i)$ for $i = 0, \ldots, k-1$ - **Local truncation error** at the $k$-th step is the difference: $$l_k = y_k - y(t_k)$$ > The error you make in just ONE step, assuming everything before was perfect. >สมมติเรารู้คำตอบจริงทั้งหมดจนถึงจุดก่อนหน้า หมายถึง $y_i = y(t_i)$ สำหรับทุก $i < k$ แล้วเราลองใช้สูตร Euler (หรือวิธีอื่น) ทำแค่ 1 step จาก $(t_k, y_k)$ → $(t_{k+1}, y_{k+1})$ ความต่างระหว่างค่าที่เราได้กับค่าจริง $y(t_{k+1})$ ของ step นั้น = Local Truncation Error --- ## Order (or Accuracy) ### Definition A numerical method is said to be of **order $p$** if: $$\boxed{l_k = O(h_k^{p+1})}$$ > Higher order = more accurate for the same step size >คำว่า “Order” หรือ “Accuracy” คือระดับความแม่นยำของวิธี บอกว่าความผิดพลาด (error) จะลดลงเร็วแค่ไหนเมื่อเรา “ลดขนาดก้าว (step size)” --- ## Order of Euler's Method ### Analysis By Taylor series: $$y(t_{k+1}) = y(t_k + h_k) = y(t_k) + y'(t_k)h_k + y''(t_k)\frac{h_k^2}{2} + \ldots$$ $$= y(t_k) + f(t_k, y(t_k))h_k + y''(t_k)\frac{h_k^2}{2} + \ldots$$ $$= y_k + f(t_k, y_k)h_k + y''(t_k)\frac{h_k^2}{2} + \ldots$$ since $y_k = y(t_k)$ by assumption of local truncation error analysis. Note that $y_k + f(t_k, y_k)h_k = y_{k+1}$. So: $$y(t_{k+1}) = y_{k+1} + y''(t_k)\frac{h_k^2}{2} + \ldots$$ $$l_{k+1} = y(t_{k+1}) - y_{k+1} = y''(t_k)\frac{h_k^2}{2} + \ldots = O(h_k^2)$$ ### Conclusion **Order of EM is 1** (2-1 มั้ง) > If you halve the step size, the error roughly halves too - not a very fast improvement. --- ## Important Theorem > [!NOTE] Theorem (Relationship Between Local and Global Error) >For a wide class of ODEs, and for a wide class of numerical integration methods (including EM): >- If the **local truncation error** is $O(h_k^{p+1})$ >- Then the **global truncation error** is $O(h_k^p)$ > Errors accumulate: one power of $h$ is "lost" when going from local to global error. --- ## Higher Order Methods: Linear Multistep Methods (LMS) >EM มันแค่ Order 1 ไม่เริ่ด อยากหา Higher Order Method บ้าง คำตอบจะได้เป๊ะ ๆ ### Motivation - We want better accuracy than Euler's method - Idea: Use information from multiple previous points ### Derivation of Adams-Bashforth Second Order (AB2) **Step 1:** Taylor series of $y(t+h)$ is: $$y(t+h) = y(t) + hy'(t) + \frac{h^2}{2}y''(t) + O(h^3)$$ $$= y(t) + hf(t, y(t)) + \frac{h^2}{2}y''(t) + O(h^3) \quad (1)$$ **Step 2:** From Taylor's series: $$y'(t-h) = y'(t) - hy''(t) + O(h^2)$$ $$f(t-h, y(t-h)) = f(t, y(t)) - hy''(t) + O(h^2)$$ **Step 3:** Rearrange the terms and multiply through by $h/2$: $$\frac{h^2}{2}y''(t) = \frac{h}{2}f(t, y(t)) - \frac{h}{2}f(t-h, y(t-h)) + O(h^3)$$ **Step 4:** Substitute into (1): $$y(t+h) = y(t) + hf(t, y(t)) + \frac{h}{2}f(t, y(t)) - \frac{h}{2}f(t-h, y(t-h)) + O(h^3)$$ $$y(t+h) = y(t) + \frac{3h}{2}f(t, y(t)) - \frac{h}{2}f(t-h, y(t-h)) + O(h^3)$$ ### Adams-Bashforth Second Order (AB2) $$\boxed{y_{k+1} = y_k + \frac{3h}{2}f(t_k, y_k) - \frac{h}{2}f(t_{k-1}, y_{k-1})}$$ **Important Note:** - $t_{k+1} - t_k = t_k - t_{k-1} = h$ - Step sizes must be the **same throughout** > AB2 looks at both where you are AND where you just came from to make a better prediction. ### Example 3: Using Adams-Bashforth Second Order #### Problem Use AB2 on the IVP in Example 2: $$\frac{dy}{dt} = \frac{t}{y}; \quad y(0) = 1$$ with constant step size $h = 0.5$ #### Solution **Initial Points:** - $(t_0, y_0) = (0, 1)$ - $(t_1, y_1) = (0.5, 1)$ **← Use EM to get this first point! (สำคัญมากกก)** **Step 1: Calculate $y_2$** $$y_2 = y_1 + \frac{3h}{2}f(t_1, y_1) - \frac{h}{2}f(t_0, y_0)$$ $$= 1 + \frac{3(0.5)}{2}\left(\frac{t_1}{y_1}\right) - \frac{0.5}{2}\left(\frac{t_0}{y_0}\right)$$ $$= 1 + \frac{1.5}{2}\left(\frac{0.5}{1}\right) - \frac{1}{4}\left(\frac{0}{1}\right) = 1.375$$ So $(t_2, y_2) = (1, 1.375)$ **Step 2: Calculate $y_3$** $$y_3 = y_2 + \frac{3h}{2}f(t_2, y_2) - \frac{h}{2}f(t_1, y_1)$$ $$= 1.375 + \frac{3(0.5)}{2}\left(\frac{1}{1.375}\right) - \frac{0.5}{2}\left(\frac{0.5}{1}\right) = 1.7955$$ So $(t_3, y_3) = (1.5, 1.7955)$ #### Comparison of Results | Method | $y(1)$ | $y(1.5)$ | | -------------- | ------------------------ | -------- | | True Solution | $\sqrt{2} \approx 1.414$ | $1.8028$ | | Euler's Method | $1.25$ | $1.65$ | | AB2 | $1.375$ | $1.7955$ | #### Conclusion **AB2 is more accurate** (as expected due to being higher order) ![[CSS322_Doable_L13_Ex3.pdf]] ## Runge-Kutta Methods (RK) ### Derivation of Runge-Kutta Method - Taylor series of $y(t+h)$ is (just like in the derivation of AB2): $$y(t+h) = y(t) + hy'(t) + \frac{h^2}{2}y''(t) + O(h^3)$$ $$y(t+h) = y(t) + hf(t,y) + \frac{h^2}{2}y''(t) + O(h^3) \quad (2)$$ #### Finding $y''(t)$ - Let's look at $y''(t)$ - Recall that: $$y'(t) = f(t,y)$$ which is the given ODE we want to solve. - Taking derivative with respect to $t$ using the chain rule yields: $$y''(t) = \frac{\partial f}{\partial t} + \frac{\partial f}{\partial y}y' = \frac{\partial f}{\partial t} + \frac{\partial f}{\partial y}f(t,y) \quad (3)$$ > Think of this as the total rate of change - how $f$ changes directly with time PLUS how it changes because $y$ is changing. ### Approximating $y''(t)$ Using Taylor Series - Next, note the Taylor series in two variables of $f(t+h, y+hf)$: $$f(t+h, y+hf) = f(t,y) + h\frac{\partial f}{\partial t} + hf(t,y)\frac{\partial f}{\partial y} + O(h^2)$$ - Rearranging: $$h\frac{\partial f}{\partial t} + hf(t,y)\frac{\partial f}{\partial y} = f(t+h, y+hf) - f(t,y) + O(h^2)$$ - Multiply through by $h/2$: $$\frac{h^2}{2}\left(\frac{\partial f}{\partial t} + f(t,y)\frac{\partial f}{\partial y}\right) = \frac{h}{2}(f(t+h, y+hf) - f(t,y)) + O(h^3)$$ - But from (3), the left-hand side of the above is: $$\frac{h^2}{2}y''(t) = \frac{h^2}{2}\left(\frac{\partial f}{\partial t} + f(t,y)\frac{\partial f}{\partial y}\right)$$ $$= \frac{h}{2}(f(t+h, y+hf) - f(t,y)) + O(h^3)$$ ### Final Formula: Heun's Method - Substituting the above into (2) yields: $$y(t+h) = y(t) + hf(t,y) + \frac{h}{2}(f(t+h, y+hf) - f(t,y)) + O(h^3)$$ $$y(t+h) = y(t) + \frac{h}{2}(f(t,y) + f(t+h, y+hf)) + O(h^3)$$ - The above gives the second-order **Runge-Kutta method** (known as **Heun's method**): $$\boxed{y_{k+1} = y_k + \frac{h}{2}(s_1 + s_2)}$$ where: - $s_1 = f(t_k, y_k)$ - $s_2 = f(t_k + h, y_k + hs_1)$ > Think of Heun's method like taking an average: you look at the slope at the start ($s_1$), use it to predict where you'd end up, check the slope there ($s_2$), then average the two slopes to take a better step. ### Example 4: Using Heun's Method #### Problem Use Heun's method on $$\frac{dy}{dt} = \frac{t}{y}; \quad y(0) = 1$$ with constant step size $h = 0.5$ #### Solution **Step 1: Find $y_1$** - $s_1 = f(t_0, y_0) = f(0, 1) = \frac{0}{1} = 0$ - $s_2 = f(t_0 + h, y_0 + hs_1) = f(0.5, 1) = \frac{0.5}{1} = 0.5$ - $y_1 = y_0 + \frac{h}{2}(s_1 + s_2) = 1 + \frac{0.5}{2}(0 + 0.5) = 1.125$ - So $(t_1, y_1) = (0.5, 1.125)$ **Step 2: Find $y_2$** - $s_1 = f(t_1, y_1) = f(0.5, 1.125) = \frac{0.5}{1.125} = 0.4444$ - $s_2 = f(t_1 + h, y_1 + hs_1) = f(1, 1.125 + (0.5)(0.4444)) = \frac{1}{1.3472} = 0.7423$ - $y_2 = y_1 + \frac{h}{2}(s_1 + s_2) = 1.125 + \frac{0.5}{2}(0.4444 + 0.7423) = 1.4217$ - So $(t_2, y_2) = (1, 1.4217)$ ![[CSS322_Doable_L13_Ex4.pdf]] --- ## Differences Between Linear Multistep (LMS) and Runge-Kutta (RK) ### What They Use - **LMS** uses $y_k, y_{k-1}, y_{k-2}, \ldots$ - Needs multiple previous points - **RK** uses only $y_k$ - Only needs current point ### Where They Evaluate $f$ - **LMS** evaluates $f$ only at $y_k$'s - Only at gridpoints - **RK** evaluates $f$ at intermediate points - Example: at $y_k + hs_1$ in Heun's method ### Trade-offs **LMS Advantages:** - Only one evaluation of $f$ per time step - More efficient per step **LMS Disadvantages:** - Harder to initialize (need multiple starting points) - Harder to change the step size (all steps must be equal) > LMS is like driving using only your rearview mirrors - you need to remember where you've been. RK is like looking ahead through the windshield at each moment. --- ## Stiffness ### Introduction to Stiffness - Consider applying Euler's method to: $$\frac{dy}{dt} = ay, \quad a < 0 \quad (4)$$ - This is the same IVP as in Example 1.2 - Recall that the true solution is: $$y(t) = y_0 e^{at}$$ - Since $a < 0$, the solution curve goes toward zero as $t$ increases ![[Pasted image 20251107120426.png|center|300]] > Like a ball rolling down a valley - all paths eventually settle to the bottom (zero), regardless of starting height. --- ### Euler's Method on the Test ODE **Applying Euler's Method:** $$y_{k+1} = y_k + ahy_k$$ $$= (1 + ah)y_k$$ $$= (1 + ah)(1 + ah)y_{k-1}$$ $$= (1 + ah)^2 y_{k-1}$$ $$\vdots$$ - Which means: $$y_k = (1 + ah)^k y_0$$ ### Stability Condition - The above means the computed solution values decay to zero as $t$ increases (same as the true solution) **only if**: $$|1 + ah| < 1$$ - Otherwise, the computed solutions $y_k$'s grow without bound, which diverges from the true solution $y(t)$ ### Deriving the Stability Range - In other words, for Euler's method to be stable, the step size $h$ must satisfy: $$|1 + ah| < 1$$ That is: $$-1 < 1 + ah < 1$$ $$-2 < ah < 0$$ $$\frac{-2}{a} > h > 0 \quad \text{(as } a < 0\text{)}$$ ### Implications - **EM is stable for the above ODE (4) only if $h < -2/a$** - ค่า $h$ ต้องเป็นเท่านี้ ถึงจะ close to true soltuion - That is, EM must take **very small time steps** - So EM is very inefficient if we are interested in $y(t)$ for large $t$ **Important Note:** Analyzing the stability and accuracy of any numerical methods on a general ODE yields essentially the same stability results as analyzing the simpler ODE (4) above! ### Definition of Stiffness An ODE is **stiff** if an explicit method (e.g., Euler's method) needs to take very small steps to maintain stability. > Stiff problems are like walking on ice - you need to take tiny, careful steps to avoid slipping (instability), even when you want to cover a long distance. --- ## Backward Differentiation Formulas (BDF) ### Introduction to BDF - **Backward Differentiation Formulas (BDF)** are commonly used methods for stiff problems - Simple BDF (first-order) is: $$\boxed{y_{k+1} = y_k + hf(t_{k+1}, y_{k+1})}$$ also known as **Backward Euler (BE)** - It is an **implicit method** - Need to solve the equation to find $y_{k+1}$ - $y_{k+1}$ appears on both sides **Why bother with BE then?** Because it has much better stability properties! > Implicit methods are like planning your route by working backwards from where you want to be, rather than just following your nose forward. --- ## Advantage of BDF Over Explicit Methods ### Stability Analysis of Backward Euler - Consider applying the Backward Euler method to the same ODE: $$\frac{dy}{dt} = ay, \quad a < 0$$ - The backward Euler method on this ODE yields: $$y_{k+1} = y_k + hay_{k+1}$$ or $$(1 - ah)y_{k+1} = y_k$$ $$y_{k+1} = \left(\frac{1}{1-ah}\right)y_k$$ ### Stability Condition for Backward Euler - So: $$y_k = \left(\frac{1}{1-ah}\right)^k y_0$$ - The backward Euler method is stable when: $$\left|\frac{1}{1-ah}\right| < 1$$ - But $a < 0$ and $h > 0$, so the denominator is always greater than 1 - This means the above holds for **any $h > 0$** ### Unconditional Stability - **BE is stable for any $h > 0$** - A method that is stable for any $h > 0$ is said to be **unconditionally stable** - In general, BDF is stable for larger range of $h$ > Backward Euler is like having stabilizers on a bike - you can go faster (take bigger steps) without worrying about falling over (instability). ## Example 5: Using Backward Euler ### Problem Use Backward Euler on the IVP $$\frac{dy}{dt} = \frac{t}{y}; \quad y(0) = 1$$ with constant step size $h = 0.5$ ### Solution **Step 1: Find $y_1$** - $(t_0, y_0) = (0, 1)$ $$y_1 = y_0 + h\frac{t_1}{y_1}$$ $$y_1 = 1 + 0.5\frac{0.5}{y_1}$$ $$y_1 = 1 + \frac{0.25}{y_1}$$ $$y_1^2 = y_1 + 0.25$$ $$y_1^2 - y_1 - 0.25 = 0$$ $$4y_1^2 - 4y_1 - 1 = 0$$ - By quadratic formula: $$y_1 = \frac{4 \pm \sqrt{(-4)^2 + 16}}{8} = \frac{1 \pm \sqrt{2}}{2}$$ - Let's take the positive root for this example: $$y_1 = \frac{1 + \sqrt{2}}{2} = 1.2071$$ (The minus root is $-0.2071$, which leads to another solution) - So $(t_1, y_1) = (0.5, 1.2071)$ ![[CSS322_Doable_L13_Ex5.pdf]] ### In Practice - In practice, we solve the nonlinear equation for $y_{k+1}$ numerically using: - Newton's method - Broyden's method [[11 - Nonlinear Equations]] - Using (perhaps) $y_k$ as the initial guess > Solving implicit equations is like solving a puzzle where you need to find a value that makes both sides equal - you iterate until you find it. --- ## The Implicit Trapezoid Method ### Motivation and Formula - BE is first-order accurate, so it is not quite useful - Let us **average** the Euler and backward Euler methods: $$\boxed{y_{k+1} = y_k + h\left(\frac{f(t_k, y_k) + f(t_{k+1}, y_{k+1})}{2}\right)}$$ - The above is the **implicit trapezoid method** >[ask] มีอีกอันคืออันนี้ อาจารย์อาจจะเหลี่ยมออกได้5555555 อาจออกได้ --- ## Stability Analysis of the Implicit Trapezoid Method ### Derivation - Applying the method to the same simple ODE (4) gives: $$y_{k+1} = y_k + h\left(\frac{ay_k + ay_{k+1}}{2}\right)$$ or $$y_{k+1} = y_k + \frac{ah}{2}y_k + \frac{ah}{2}y_{k+1}$$ $$y_{k+1} - \frac{ah}{2}y_{k+1} = y_k + \frac{ah}{2}y_k$$ $$\left(1 - \frac{ah}{2}\right)y_{k+1} = \left(1 + \frac{ah}{2}\right)y_k$$ $$y_{k+1} = \left(\frac{1 + ah/2}{1 - ah/2}\right)y_k$$ ### Stability Result - So: $$y_k = \left(\frac{1 + ah/2}{1 - ah/2}\right)^k y_0$$ - The implicit trapezoid method is stable when: $$\left|\frac{1 + ah/2}{1 - ah/2}\right| < 1$$ - Since $a < 0$, the above holds for **any $h > 0$** - That is, the implicit trapezoid method is stable for any $h > 0$ - It is **unconditionally stable** > The trapezoid method combines the best of both worlds - second-order accuracy AND unconditional stability! >Only suitable for stiff problem! --- ## Solving a System of Coupled ODEs ### General Form - All of the methods covered can also be used on a system of coupled first-order ODEs: $$\mathbf{y}' = \mathbf{f}(t, \mathbf{y})$$ where $\mathbf{f}: \mathbb{R}^{n+1} \to \mathbb{R}^n$, or in full: $$\mathbf{y}'(t) = \begin{bmatrix} dy_1(t)/dt \\ dy_2(t)/dt \\ \vdots \\ dy_n(t)/dt \end{bmatrix} = \begin{bmatrix} f_1(t,\mathbf{y}) \\ f_2(t,\mathbf{y}) \\ \vdots \\ f_n(t,\mathbf{y}) \end{bmatrix} = \mathbf{f}(t,\mathbf{y})$$ > Think of a system of ODEs like tracking multiple related quantities simultaneously - position AND velocity, or predator AND prey populations. --- ## Example 6: System of ODEs with Euler's Method ### Problem Perform two steps of Euler's method on the IVP: $$\mathbf{y}' = \begin{bmatrix} y_1' \\ y_2' \end{bmatrix} = \begin{bmatrix} ty_2 - y_1 \\ 2y_1^2 y_2 + t^2 \end{bmatrix}, \quad \mathbf{y}(0) = \begin{bmatrix} -2 \\ 1 \end{bmatrix}$$ using constant step size $h = 0.2$ ### Solution **Step 0:** - $t_0 = 0$ - $\mathbf{y}_0 = \begin{bmatrix} -2 \\ 1 \end{bmatrix}$ **Step 1: Find $\mathbf{y}_1$** - $t_1 = t_0 + h = 0.2$ $$\mathbf{y}_1 = \mathbf{y}_0 + h\mathbf{f}(t_0, \mathbf{y}_0)$$ $$= \begin{bmatrix} -2 \\ 1 \end{bmatrix} + (0.2)\begin{bmatrix} 0(1) - (-2) \\ 2(-2)^2(1) + 0^2 \end{bmatrix} = \begin{bmatrix} -1.6 \\ 2.6 \end{bmatrix}$$ **Step 2: Find $\mathbf{y}_2$** - $t_2 = t_1 + h = 0.4$ $$\mathbf{y}_2 = \mathbf{y}_1 + h\mathbf{f}(t_1, \mathbf{y}_1)$$ $$= \begin{bmatrix} -1.6 \\ 2.6 \end{bmatrix} + (0.2)\begin{bmatrix} (0.2)(2.6) - (-1.6) \\ 2(-1.6)^2(2.6) + (0.2)^2 \end{bmatrix} = \begin{bmatrix} -1.176 \\ 5.2704 \end{bmatrix}$$ ![[CSS322_Doable_L13_Ex6.pdf]] --- ## Solving a Higher-Order ODE >Method ที่เราพูดกันมา จริง ๆ มันก็ Solve Higher-Order ได้ ไม่จำเป็นต้อง First-Order ตลอดไป >Any higher order ODE can be transformed into an equivalent first-order ได้ด้วยอะ ### Key Concept - We can use the discussed methods to solve a **higher-order ODE** (an ODE with higher-order derivatives) - A higher-order ODE can always be transformed into an equivalent first-order ODE system --- ## Example 7: Newton's Second Law ### Original Equation Newton's second law: $$F = ma = my''$$ or $$y'' = F/m$$ ### Transformation to First-Order System - Define the new unknowns: - $u_1(t) = y(t)$ (position) - $u_2(t) = y'(t)$ (velocity) - We can transform Newton's second law to the system of first-order ODEs: $$\begin{bmatrix} u_1' \\ u_2' \end{bmatrix} = \begin{bmatrix} u_2 \\ F/m \end{bmatrix}$$ > This is like separating motion into position and velocity - each has its own equation, but they're linked together. --- ## General Transformation: $k$-th Order ODE to First-Order System ### General Form The $k$-th order ODE of the form: $$y^{(k)} = f(t, y, y', y'', \ldots, y^{(k-1)})$$ can be transformed into: $$\mathbf{u}' = \begin{bmatrix} u_1' \\ u_2' \\ \vdots \\ u_{k-1}' \\ u_k' \end{bmatrix} = \begin{bmatrix} u_2 \\ u_3 \\ \vdots \\ u_k \\ f(t, u_1, u_2, \ldots, u_k) \end{bmatrix} = \mathbf{g}(t, \mathbf{u})$$ where: - $u_1(t) = y(t)$ - $u_2(t) = y'(t)$ - $\vdots$ - $u_k(t) = y^{(k-1)}(t)$ > Each variable represents one level of derivative - position, velocity, acceleration, jerk, etc. - creating a cascade of relationships. --- ### Exercises 7 #### Problems Write the following ODEs as an equivalent first-order system of ODEs: - **Problem 1:** $y'' = t + y + y'$ - **Problem 2:** $y''' = y'' + ty$ #### Solutions ##### Solution 1: $y'' = t + y + y'$ Let $u_1(t) = y(t)$, $u_2(t) = y'(t)$ $$\boxed{\begin{bmatrix} u_1' \\ u_2' \end{bmatrix} = \begin{bmatrix} u_2 \\ t + u_1 + u_2 \end{bmatrix}}$$ ##### Solution 2: $y''' = y'' + ty$ Let $u_1(t) = y(t)$, $u_2(t) = y'(t)$, $u_3(t) = y''(t)$ $$\boxed{\begin{bmatrix} u_1' \\ u_2' \\ u_3' \end{bmatrix} = \begin{bmatrix} u_2 \\ u_3 \\ u_3 + tu_1 \end{bmatrix}}$$ ![[CSS322_Doable_L13_Ex7 1.pdf]] --- ## Summary: Comparison of Methods ### Method Overview Table | Method | Order | Type | Stability | Evaluations per Step | Initialization | |--------|-------|------|-----------|---------------------|----------------| | Euler | 1 | Explicit, One-step | Conditional ($h < -2/a$) | 1 | Easy | | Adams-Bashforth 2 | 2 | Explicit, Multistep | Conditional | 1 | Requires previous point | | Heun (RK2) | 2 | Explicit, One-step | Conditional | 2 | Easy | | Backward Euler | 1 | Implicit, One-step | Unconditional | 1 (+ solve) | Easy | | Implicit Trapezoid | 2 | Implicit, One-step | Unconditional | 2 (+ solve) | Easy | ### Key Takeaways 1. **Explicit vs Implicit** - Explicit: Easy to compute, but stability constraints - Implicit: Harder to compute (need to solve equations), but better stability 2. **One-step vs Multistep** - One-step: Easier to start, easier to change step size - Multistep: More efficient per step, harder to initialize 3. **Order Matters** - Higher order = more accurate for same step size - Order 1 methods generally not recommended 4. **Stiffness** - Explicit methods struggle with stiff problems - Implicit methods (BDF, trapezoid) handle stiffness well 5. **Runge-Kutta Methods** - Intermediate evaluations improve accuracy - One-step makes them flexible - Trade: more function evaluations per step > Choosing a method is like choosing a vehicle: sports car (explicit) is fast on smooth roads, but a truck (implicit) handles rough terrain (stiffness) better, even if it's slower on straightaways. --- ## Practical Recommendations ### For Non-Stiff Problems - Use Runge-Kutta methods (RK4 is popular) - Or Adams-Bashforth multistep methods for efficiency ### For Stiff Problems - Use BDF methods - Or implicit trapezoid method - Accept the cost of solving implicit equations ### General Advice - Start with order 2 or higher methods - Avoid first-order methods unless absolutely necessary - Modern software often uses adaptive step-size control - For production code, use well-tested libraries (MATLAB's ode45, Python's scipy.integrate)