🧪 CSS323 — Software Quality Assurance (SQA) Cheat Sheet

1. What Is Quality?

One-liner: Quality = the characteristic that distinguishes the grade of excellence or superiority of a process, product, or service.

• Quality means different things in different contexts — it's not just "no bugs"
• In software: quality covers functionality, reliability, usability, maintainability, and more

🍎 Apple example: A product can work perfectly and still have poor quality — early Apple Maps (iOS 6) technically functioned, but quality was poor because it gave wrong directions. Quality = meeting what users actually need, not just what was specified.

2. The Quality Trilogy

Three interconnected pillars — Plan → Control → Improve

📋 Quality Planning

Define the target before you build:

• Identify the customer and their needs
• Develop product features to meet those needs
• Establish quality goals
• Develop a process to deliver them
• Prove process capability (can this process actually hit the goals?)

🔍 Quality Control

Monitor during execution:

• Choose control subjects (what to measure)
• Choose units of measurement
• Establish standards for performance
• Measure actual performance
• Interpret the difference (actual vs. standard)
• Take action on the difference

📈 Quality Improvement

Fix root causes, not symptoms:

• Prove the need for improvement (with data)
• Identify specific projects to improve
• Diagnose root causes
• Provide remedies and prove they work under real conditions
• Hold the gains (prevent regression)

🍎 Apple analogy:

• Planning = writing the feature spec and test plan for a new iOS API
• Control = running automated unit tests and CI checks on every PR
• Improvement = analyzing crash reports in Xcode Organizer and fixing root causes, not just symptoms

3. ISO 9126 Quality Model (1991)

Six characteristics — the "six sides of the die" — all must be balanced:

⚠️ A product that scores high on Functionality but low on Usability is still poor quality — all 6 sides matter.

4. Software Quality Assurance (SQA)

Foundation Principle

Quality is measured by conformance to requirements — both explicit and implicit.

Three Core Concepts

🍎 QC vs QA in Apple terms:

• QC = a tester finding that the camera crashes when zoom > 5x before release
• QA = the process that ensures camera code is always reviewed by 2 engineers + passes 300 automated tests before merging

5. Variation Control

Variation control = the heart of quality control

Software engineers control variation in three areas:

1. The process applied
2. Resources expended
3. End product quality attributes

The goal is to make quality predictable — not just hoped for.

6. Quality Costs

Fixing bugs early is dramatically cheaper than fixing them in production.

🍎 Apple example: A 1-star App Store review that says "crashes on launch" = External Failure cost. It costs far more to repair reputation + issue a hotfix + respond to reviews than it would have cost to write proper unit tests during development.

💡 Rule of thumb: The cost of fixing a bug multiplies by ~10x at each stage: Design → Code → Test → Release → Post-release.

7. SQA Group — 6 Activities

1. Prepare the SQA Plan for the project
2. Participate in developing the project's software process description
3. Review software engineering activities to verify compliance with defined process
4. Audit designated work products to verify compliance
5. Ensure deviations in software or work products are documented and handled per procedure
6. Record noncompliance evidence and report to management

8. Software Reviews

Purpose: Find defects before they are passed to the next activity or shipped to the customer.

Formal Technical Review (FTR) = also called a walkthrough or inspection. One of the most effective QA tools.

Review Roles

FTR Guidelines (Memorize These Numbers!)

🍎 Apple analogy: Code review on GitHub. The PR author walks through the changes (Presenter), the tech lead facilitates (Coordinator), comments are recorded in GitHub (Recorder), and other engineers review (Reviewers). Same structure.

Why Do Peer Reviews?

• Catches ~80% of all errors if done properly
• Catches both coding errors AND design errors (early!)
• Enforces organizational standards
• Provides training and team knowledge insurance

Formality vs. Timing Trade-off

When to Perform Reviews

• When analysis is complete
• When design is complete
• After first compilation
• After first test run
• After all test runs
• Any time a complete work product is produced

Review Conduct Guidelines

• Keep it short (< 30 minutes per session)
• Don't schedule two reviews back-to-back
• Don't review fragments — review complete work products
• Use standards to avoid style disagreements
• Let the coordinator run and maintain order

9. Formal SQA Approaches

Two main approaches:

1. Proof of Correctness — mathematical/logical proof that code meets spec
2. Statistical Quality Assurance — data-driven defect tracking

Statistical Quality Assurance Process

1. Collect and categorize information about software defects
2. Trace each defect back to its root cause
3. Apply Pareto Principle (80/20 Rule):

80% of defects can be traced to just 20% of the causes → fix the "vital few"

4. Correct the vital few causes

🍎 Apple example: Xcode Organizer's crash analytics. You don't fix every crash — you sort by frequency and fix the top 3–5 crash types that account for 80% of all crashes first. That's Pareto applied to software quality.

10. Software Reliability

Definition: The probability of failure-free operation of a computer program in a specified environment for a specified time period.

• Can be measured directly and estimated using historical + developmental data
• Reliability problems usually trace back to errors in design or implementation

Reliability Metrics (Know All Four!)

Key Formulas

$\boxed{Availability = \frac{MTBF}{MTBF + MTTR}}$

$\boxed{Reliability = \frac{MTBF}{1 + MTBF}}$

Where:

• $MTBF$ = Mean Time Between Failure
• $MTTR$ = Mean Time to Repair

🍎 Apple example: iCloud availability is measured this way. If iCloud goes down for 1 hour every 999 hours of operation: MTBF = 999, MTTR = 1 → Availability = 999/1000 = 99.9% ("three nines").

Time Units for Reliability Measurement

11. Software Safety

Safety focuses on identifying potential hazards that could cause a system to fail with consequences.

• Reliability = likelihood of failure
• Safety = impact/consequences of failure

🍎 Apple example: A bug in Apple Maps is a reliability issue. A bug in an iPhone's autonomous driving integration that causes an accident is a safety issue. Same code base, different stakes.

Validation Perspectives — Three Types

Validation Techniques

Safety Review Checklist

• Are the intended system functions correct?
• Is the structure maintainable and understandable?
• Verify algorithm and data structure design against spec
• Check code consistency with algorithm/data structure design
• Review adequacy of system testing

Hazard-Driven Analysis

Safety assurance relies on hazard identification first. Three approaches:

12. SQA Plan — 7 Sections

13. PSP / TSP / CMMI Overview

They stack: PSP builds disciplined individuals → TSP uses those individuals to build effective teams → CMMI scales team practices across the whole organization.

14. PSP — Personal Software Process

PSP = a framework to help individual engineers plan, measure, and improve their own work using data.

PSP Process Flow (8 Steps — in order)

Planning → Design → Design Review → Code → Code Review → Compile → Test → Postmortem
                                                                              ↓
                                                                      Finished Product

Supporting throughout:

• Scripts — guide what to do at each step
• Logs — record time and defects at each step
• Plan Summary → Project & Process Data Summary Report

PSP Phases & Activities

PSP Inputs Required

• Problem description
• PSP Project Plan Summary form
• Historical estimated AND actual size/time data
• Time and Defect Recording Logs
• Defect Type Standard

PSP Exit Criteria (Must complete ALL before "done")

• Thoroughly tested program
• Completed Project Plan Summary
• Completed design templates
• Completed Design Review & Code Review Checklists
• Completed Test Report Template
• Completed PIP (Process Improvement Proposal) forms
• Completed Defect and Time Recording Logs

15. TSP — Team Software Process

TSP extends PSP to teams — enabling groups to build quality products within cost and schedule.

TSP Goals

• Build teams quickly and reliably
• Optimize team performance throughout a project
• Accelerate software process improvement
• Make mature processes normal and expected

TSP ≈ CMM/CMMI maturity level 5 process — for teams.

TSP Strategy (Bottom-Up Layers)

16. CMMI — Capability Maturity Model Integration

CMMI = A collection of characteristics of effective processes that guides organizations in improving how they develop and manage products/services.

What CMMI Helps Organizations Do:

• Examine effectiveness of current processes
• Establish priorities for improvement
• Implement improvements systematically

The 5 Maturity Levels (Must Know All 5!)

🍎 Apple analogy:

• Level 1: A solo dev who ships features when they "feel ready"
• Level 2: A startup with a basic Jira board and weekly sprint reviews
• Level 3: A company with documented engineering processes, defined code review rules, and onboarding guides
• Level 4: A company measuring defect rates, deployment frequency, and MTTR with dashboards
• Level 5: Apple's own release pipeline — automated regression testing, statistical process monitoring, CI/CD, and retrospectives feeding back into process updates

17. CMMI Models — Three Constellations

All three share 16 Core Process Areas:

18. CMMI-DEV Process Areas by Maturity Level

19. CMMI Process Areas by Category

20. Process Area (PA) Components

Each Process Area has three component types:

21. CMMI Model Structure (Pyramid)

        ┌────────────────────────────────────┐
        │  BENCHMARK RATINGS                 │  ← Goals, PAs, Maturity Levels, Capability Levels
        ├────────────────────────────────────┤
        │  CONSTELLATION-SPECIFIC PAs        │  ← DEV / SVC / ACQ specific process areas
        ├────────────────────────────────────┤
        │  CMMI MODEL FOUNDATION             │  ← 16 Core PAs (shared across all constellations)
        ├────────────────────────────────────┤
        │  INSTITUTIONALIZATION              │  ← Policies, Plans, Resources, Training, Monitoring
        └────────────────────────────────────┘  ← FOUNDATION

🍎 Apple codebase analogy:

• Institutionalization = AppDelegate, project config, CI/CD pipeline (infrastructure)
• Core PAs = Shared frameworks used across all teams (Networking, Analytics, UI Kit)
• Constellation PAs = Feature-specific modules (Maps team, Siri team, Camera team)
• Benchmark Ratings = App Store metrics, crash-free rate dashboards

22. Big Picture Summary

QUALITY TRILOGY
   Planning → Control → Improvement (continuous loop)

ISO 9126 — 6 Characteristics
   Functionality · Reliability · Usability · Efficiency · Maintainability · Portability

SQA TOOLS
   ├── Formal Technical Reviews (FTR) — 3–5 people, <2hrs, catches ~80% of errors
   ├── Statistical QA — Pareto Principle (80% defects from 20% causes)
   └── SQA Plan — 7 sections (Mgmt, Docs, Standards, Reviews, Test, Problem Reporting, Other)

RELIABILITY METRICS
   POFOD · ROCOF · MTTF/MTBF
   Availability = MTBF / (MTBF + MTTR)
   Reliability  = MTBF / (1 + MTBF)

SAFETY = Reliability + Consequences
   Hazard Avoidance → Accident Avoidance → Protection Systems

PROCESS FRAMEWORKS (Individual → Team → Org)
   PSP (8-step: Plan→Design→DR→Code→CR→Compile→Test→Postmortem)
    ↓
   TSP (Goal setting, Team building, Management)
    ↓
   CMMI (5 Levels: Initial→Managed→Defined→Quantitative→Optimizing)
         (3 Models: DEV · SVC · ACQ)
         (4 Categories: Process Mgmt · Support · Project Mgmt · Engineering)

🍎 Final thought: Apple is widely considered a Level 5 organization — not because they never have bugs, but because when a bug appears, their process automatically learns from it, documents it, and prevents it from happening again. That's what CMMI Level 5 actually means: improvement isn't a project, it's a habit.

Cheat sheet for CSS323 Software Engineering — Software Quality Assurance Chapter Quality isn't tested in — it's built in. 🔬