8.1 Probabilities

• A probability formally measures uncertainty by representing the degree of belief (0…1)
• A random variable (ตัวแปรเชิงสุ่ม) is a variable in probability theory with:
  • A domain of possible values
  • An associated probability distribution

Examples of Random Variables

Toothache Example:

• Variable: 'a person has toothache'
• Domain: {true, false} → Boolean random variable
• Probabilities:
  • $p(\text{Toothache} = \text{true}) = p(\text{toothache}) = 0.2$
  • $p(\text{Toothache} = \text{false}) = p(\neg\text{toothache}) = 0.8$
  • $\mathbf{P}(\text{Toothache}) = \langle 0.2, 0.8 \rangle$
    • $\mathbf{P}$ เรียกว่า Probability distribution - collection of probability of all possible case
    • $\sum$ รวมกันทั้งหมดก็ต้องเป็น 1 ถูกมะ!!

เขียนได้สองแบบ ปกติกับ Short hand: $\text{Toothache} = \text{true}\equiv \text{toothache}$

Weather Example:

• Variable: 'a day's weather'
• Domain: {sun, rain, cloud, snow}
• Probabilities:
  • $p(\text{Weather} = \text{sun}) = 0.6$
  • $p(\text{Weather} = \text{rain}) = 0.1$
  • $p(\text{Weather} = \text{cloud}) = 0.29$
  • $p(\text{Weather} = \text{snow}) = 0.01$
  • $\mathbf{P}(\text{Weather}) = \langle 0.6, 0.1, 0.29, 0.01 \rangle$
    • เช่นเดียวกัน รวมกัน $0.6+0.1+0.29+0.01=1$

Basic Probability Rules

Two types or probability

Unconditional and Conditional Probability

• Unconditional probability $p(X)$: degree of belief without any other information
• Conditional probability $p(X | Y)$: likelihood of $X$ when ==$Y$ is evidence== ($Y$ is known!)
  • เวลาอ่าน ให้อ่านว่า “$X$ given $Y$”

Key Formulas

Conditional Probability:
$\boxed{p(X | Y) = \frac{p(X \land Y)}{p(Y)}=\frac{p(X \cap Y)}{p(Y)} = \frac{p(X, Y)}{p(Y)}}$
(when $p(Y) > 0$)

Product Rule:
$\boxed{p(X, Y) = p(X | Y)p(Y)}$
$\boxed{p(X, Y) = p(Y | X)p(X)}$

Note: We use $p(X, Y)$ to denote $p(X \cap Y)$

For Three Variables:
$\boxed{p(X, Y, Z) = p(X|Y,Z)p(Y|Z)p(Z)}$
หรือ
$\boxed{p(X, Y, Z) = p(Z|X,Y)p(Y|X)p(X)}$

• เช็คอีกทีว่าถูกจริงเหรอ

Chain Rule

For n variables:
$\boxed{p(X_n, X_{n-1},...,X_2, X_1) = \prod_{i=1}^{n} p(X_i | X_{i-1},...,X_1)}$

Expanded form:
$= p(X_n | X_{n-1},...,X_1)p(X_{n-1} | X_{n-2},...,X_1) \cdots p(X_2 | X_1)p(X_1)$

Boolean Variables Rules

For Boolean random variables $A$ and $B$:

• Negation: $p(\neg a) = 1 - p(a)$ or $p(A = \text{false}) = 1 - p(A = \text{true})$
• Inclusion-Exclusion Principle:
  $\boxed{p(A \lor B) = p(A) + p(B) - p(A, B)}$

Marginalization (Summing Out)

• Given a full joint probability distribution of all combinations of values
• The probability over a subset can be computed by summing out unwanted variables

Examples with Three Boolean Variables (A, B, C)

ถ้าเป็น 3 Boolean Variables อย่าง A, B, C → ก็มีได้ทั้งหมด 8 cases ถูกป่าว ทั้ง 8 อันรวมกัน เรียกว่า Full joint probability distrubution!

Marginalizing over C:
$p(A, B) = p(A, B, c) + p(A, B, \neg c) = \sum_{C \in \{\text{true}, \text{false}\}} p(A, B, C)$

Marginalizing over A and C:
$p(B) = p(a, B, c) + p(a, B, \neg c) + p(\neg a, B, c) + p(\neg a, B, \neg c)$
$= \sum_{A \in \{\text{true}, \text{false}\}} \left[ \sum_{C \in \{\text{true}, \text{false}\}} p(A, B, C) \right]$

Compact notation: $\sum_{C \in \{\text{true}, \text{false}\}} p(A, B, C)$ can be written as $\sum_C p(A, B, C)$

Example 8.1: Full Joint Distribution

Given a full joint distribution for Toothache, Cavity, Catch:

Find the following probability values:

1. $p(\text{cavity})$ = [to be calculated]

2. $p(\text{cavity} \cup \text{toothache})$ = [to be calculated]

3. $p(\text{catch} \cap \neg\text{cavity})$ = [to be calculated]

4. $p(\text{cavity} | \text{toothache})$ = [to be calculated]

5. $p(\text{catch} | (\text{toothache} \cap \text{cavity}))$ = [to be calculated]

8.2 Bayes' Rule

Bayes' rule flips a conditional probability:

$\boxed{p(Y | X) = \frac{p(X | Y)p(Y)}{p(X)}}$

Expanded form:
$p(Y | X) = \frac{p(X | Y)p(Y)}{p(X, y) + p(X, \neg y)} = \frac{p(X | Y)p(Y)}{p(X | y)p(y) + p(X | \neg y)p(\neg y)}$

Independence

Random variables $X$ and $Y$ are independent if and only if:
$\boxed{p(X, Y) = p(X)p(Y)}$

Example 8.2: Tuberculosis Test

Problem:

• Reliability of skin test for active pulmonary tuberculosis (TB):
  • Of people with TB: 98% positive reaction, 2% negative reaction
  • Of people without TB: 99% negative reaction, 1% positive reaction
• From a large population where 2 per 10,000 persons have TB
• A person is selected at random, given a skin test → positive result
• Question: What is the probability the person has active pulmonary TB?

Solution: [Space left for working]

8.3 Bayesian Networks

A Bayesian Network is a directed acyclic graph representing a probabilistic graphical model.

Definition

A Bayesian Network $B = (G, \Theta)$ where:

• $G$ is a directed graph with no cycles
  • Each vertex $X_i$ represents a random variable
  • Each edge from $X_i$ to $X_j$ represents statistical dependency (between two variables น้า)
    • $X_i$ is a parent
    • $X_j$ is a child
• $\Theta$ represents the set of parameters specifying probability distributions for each variable

Example Structure

• Parents of X: A, B
• Children of X: C, D
• Conditional probabilities: $p(X|A)$, $p(X|B)$, $p(C|X)$, $p(D|X)$

8.4 Conditional Independence

$X$ and $Y$ are independent, if the value of $Z$ is known

Definition: Variables $X$ and $Y$ are conditionally independent given $Z$ if and only if:
$\boxed{p(X, Y | Z) = p(X | Z)p(Y | Z)}$

Equivalent forms:

• $p(X | Y, Z) = p(X | Z)$
• $p(Y | X, Z) = p(Y | Z)$

Joint Probability in Bayesian Networks

General chain rule:
$p(X_1, ..., X_n) = \prod_{i=1}^{n} p(X_i | X_1, ..., X_{i-1})$
In Bayesian Networks:

• Each $X_i$ is conditionally independent of its ancestors given its parents
• Therefore:
  $\boxed{p(X_1, ..., X_n) = \prod_{i=1}^{n} p(X_i | \text{parents of } X_i)}$

Example 8.3: Joint Probability Distributions

Write the joint probability distribution for the following Bayesian networks:

ทำง่าย ๆ เลยล่ะ แค่เอาแม่ของมันมาเขียนไว้ข้างหลัง ของแต่ละตัว (ระวัง หลายตัวมีแม่หลายคนนะโกโก้)

1. Network with B, A, C, D, E

Answer: $\mathbf{P}(A,B,C,D,E)=p(A)p(B)p(C|B)p(D|A,C)p(E|D)$

• IF! Without the Bayesian Network $\mathbf{P}(A,B,C,D,E)=p(A)p(B|A)p(C|A,B)p(D|A,B,C)p(E|A,B,C,D)$

2. Network with A, B, C, D

Answer: $\mathbf{P}(A,B,C,D)=p(A)p(B|A)p(C|A)p(D|B,C)$

3. Chain Network

Answer: $\mathbf{P}(A,B,C,D)=p(A)p(B|A)p(C|B)p(D|C)$

8.5 Common Causes (Tail-to-Tail)

• Vertex $C$ is tail-to-tail w.r.t. the path from $A$ to $B$

Probability Relations

$p(A, B, C) = p(A|C)p(B|C)p(C)$

$p(A, B) = \sum_{c \in C} p(A|c)p(B|c)p(c) \neq p(A)p(B)$

$p(A, B|C) = \frac{p(A|C)p(B|C)\cancel{p(C)}}{\cancel{p(C)}} = p(A|C)p(B|C)$

Conclusion

• $A$ and $B$ are NOT independent
• $A$ and $B$ ARE conditionally independent given $C$

8.6 Causal Chain (Head-to-Tail)

• Vertex $C$ is head-to-tail w.r.t. the path from $A$ to $B$

Probability Relations

$p(A, B, C) = p(A)p(C|A)p(B|C)$

$p(A, B) = \sum_{c \in C} p(A)p(c|A)p(B|c) \neq p(A)p(B)$

$p(A, B|C) = \frac{p(A)p(C|A)p(B|C)}{p(C)} = p(A|C)p(B|C)$

• แอบใช้ Bayes’ rule นะ $\color{gray}\frac{p(A)p(C|A)}{p(C)}=p(A|C)$

Conclusion

• $A$ and $B$ are NOT independent
• $A$ and $B$ ARE conditionally independent given $C$

8.7 Common Effects (Head-to-Head)

• Vertex $C$ is head-to-head w.r.t. the path from $A$ to $B$

Probability Relations

$p(A, B, C) = p(A)p(B)p(C|A, B)$
$p(A, B) = \sum_{c \in C} p(A)p(B)p(c|A, B) = p(A)p(B) \sum_{c \in C} p(c|A, B) = p(A)p(B)$
$p(A, B|C) = \frac{p(A)p(B)p(C|A, B)}{p(C)} \neq p(A|C)p(B|C)$

Conclusion

• $A$ and $B$ ARE independent
• $A$ and $B$ are NOT conditionally independent given $C$

Example 8.4: Burglar Alarm Network

Scenario:

• You install a new burglar alarm
• It reliably detects burglary but also responds to minor earthquakes
• Two neighbors (John and Mary) promise to call police when they hear alarm
• John: Always calls when hears alarm, but sometimes confuses alarm with phone ringing
• Mary: Likes loud music, sometimes doesn't hear alarm
• Goal: Estimate probability of burglary given evidence about who called

Bayesian Network Structure

Probability Tables

Prior Probabilities:

Alarm given Burglary and Earthquake:

John Calls given Alarm:

Mary Calls given Alarm:

เรารู้ข้างบนแล้วสามารถคำนวณสิ่งนี้ได้ → $p(B,E,A,J,M)=p(B)p(E)p(A|B,E)p(J|A)p(M|A)$

Questions

1. What is the probability that the alarm has sounded but neither burglary nor earthquake has occurred, and both Mary and John call?

2. What is the probability that the alarm would sound, given that there was an earthquake?

3. What is the probability that the alarm will sound?

4. What is the probability that there was a burglary, given that John called?

[Space for solution]