🧪 CSS323 — Chapter 8: Software Testing Cheat Sheet

1. What Is Program Testing?

One-liner: Testing = executing a program with artificial data to find defects before it goes into use.

• Checks results for errors, anomalies, or non-functional attributes
• Testing can reveal the presence of errors — NOT their absence
• Testing is part of the larger Verification & Validation (V&V) process

🍎 Apple analogy: When Apple tests Face ID, they can confirm it fails under certain lighting conditions (defect found!), but they can never guarantee it works perfectly in every possible future situation. That's why testing never fully ends.

🍽️ Food analogy: Testing is like tasting food while cooking — you can find out something tastes wrong, but tasting once doesn't guarantee every bite will be perfect.

2. Two Goals of Testing

Goal 1 — By Software Type:

Validation vs. Defect Testing

⚠️ Major defects must be filtered out before release — defect testing is adversarial by design.

3. Verification & Validation (V&V)

V&V Confidence Depends On:

1. Software purpose — safety-critical systems need higher confidence
2. User expectations — some software types have naturally lower user expectations
3. Marketing environment — getting to market fast may outweigh finding all defects

🍎 Apple example: Apple Watch ECG feature = very high V&V confidence needed (safety-critical). A widget that shows a fun quote = much lower threshold.

4. Inspections vs. Testing

Inspection Targets:

• Requirements specification
• Software architecture
• UML design models
• Database schemas
• Program source code

Advantages of Inspections Over Testing:

• Errors don't mask other errors (static = no interaction effects)
• Can inspect incomplete versions without test harnesses
• Can check portability, maintainability, standards compliance — things tests can't easily check

⚠️ They're Complementary — Use BOTH:

• Inspections: good for checking spec conformance, bad for non-functional attributes (performance, usability)
• Testing: good for behavioral/runtime validation

5. Software Testing Process (5 Steps)

1. Design test cases  →  Test Cases
2. Prepare test data  →  Test Data
3. Run program        →  Test Results
4. Compare results    →  Test Reports
5. Loop back if needed

6. Stages of Testing (The Big 3)

7. Development Testing — Three Levels

System Testing       ← test the whole integrated system
    ↑
Component Testing    ← test composite components (interface between units)
    ↑
Unit Testing         ← test individual functions/methods/classes in isolation

8. Unit Testing

Testing individual components in isolation — this is defect testing.

Units can be:

• Individual functions or methods within an object
• Object classes with multiple attributes and methods
• Composite components with defined interfaces

Object Class Testing — Complete Coverage Requires:

• Testing all operations associated with an object
• Setting and interrogating all object attributes
• Exercising the object in all possible states

🍎 Apple example: Testing CLLocationManager in isolation — verify it correctly starts/stops location updates, handles permission denied, handles GPS unavailable — all states, all methods, before integrating with the rest of the app.

9. Automated Testing

Unit testing should be automated wherever possible.

3 Components of an Automated Test

• true → test passed ✅
• false → test failed ❌

🍎 Xcode/XCTest is Apple's test automation framework — same concept as JUnit for Java.

10. Choosing Unit Test Cases

Two types of test cases:

11. Testing Strategies

Strategy 1 — Partition Testing (Equivalence Partitioning)

Identify groups of inputs (equivalence partitions) that should be processed the same way — test one value from each group.

• Each group = equivalence partition or domain
• No need to test every value — just one representative per partition

Example — Number of Input Values:

Example — Input Values:

🍎 Apple example: Testing App Store age rating input: test "4" (valid child), "17" (valid adult), "0" (invalid too low), "200" (invalid too high) — 4 partitions, 4 tests. You don't test every age from 0–200.

🧮 Calculator analogy: No need to test 2×3 AND 2×4 separately — both are "normal multiplication." Same partition, one test is enough.

Strategy 2 — Guideline-Based Testing

Use guidelines from previous experience of common programmer errors

12. Testing Guidelines

For Sequences:

• Test with sequences that have only a single value
• Use sequences of different sizes
• Access the first, middle, and last elements
• Test with sequences of zero length

General:

• Force system to generate all error messages
• Design inputs that cause input buffer overflow
• Repeat the same input many times
• Force invalid outputs
• Force computation results to be too large or too small

13. Component Testing

Testing composite components made of several interacting objects — accessed through the defined component interface.

• Assumes unit tests on individual objects are already completed
• Focus: show the component interface behaves per specification

Interface Types

Interface Errors (3 Types — Know All!)

Interface Testing Guidelines:

• Test parameters at extreme ends of their ranges
• Test pointer parameters with null pointers
• Design tests that cause the component to fail
• Use stress testing in message passing systems
• In shared memory systems, vary the activation order of components

14. System Testing

Integrating components and testing the interactions between them — testing emergent behavior.

Checks components are:

• Compatible
• Interact correctly
• Transfer the right data at the right time across interfaces

System Testing Key Facts:

• Reusable/off-the-shelf components may be included
• Components from different team members are integrated here
• System testing is collective — not individual
• Some organizations use a separate testing team with no designer/programmer involvement

Use-Case Testing

• Use-cases = basis for system test design (use case diagram = functional requirements → test cases)
• Each use case involves several system components → forces interactions to occur
• Sequence diagrams document which components and interactions are being tested

Example — Weather Station test cases from sequence diagram:

• A report request should have an acknowledgement and a report should be returned
• Create summarized data → check the report is correctly organized
• Create raw data → check WeatherStation correctly produces the summary

15. Testing Policies

Exhaustive system testing is impossible — you need policies to decide what to test.

Examples:

• All functions accessed via menus must be tested
• Combinations of functions from the same menu must be tested
• For any user input: test with both correct AND incorrect input

16. Test-Driven Development (TDD)

Write the test first, then write the code to pass it. Tests drive development.

TDD Process (5 Steps — in order)

1. Identify small increment of functionality needed
2. Write an automated test for it
3. Run the test → it FAILS (expected — code doesn't exist yet) ❌
4. Implement the functionality → re-run test
5. Test PASSES ✅ → move on to next increment

🍎 Apple/Swift example:

// Step 1-2: Write the test FIRST
func testFormatCurrency() {
    XCTAssertEqual(formatCurrency(1234.5), "฿1,234.50")
}
// Step 3: Run → FAILS (formatCurrency doesn't exist yet)
// Step 4: Implement formatCurrency()
// Step 5: Run → PASSES ✅ → move to next feature

TDD Benefits (Know All 4!)

17. Regression Testing

Testing the system to check that changes have not broken previously working code.

• Tests must pass before a change is committed

📝 Analogy: Like spell-checking a whole document every time you add a new paragraph — your new edits shouldn't break the old text.

🍎 Apple example: Every time an engineer pushes code to the iOS repo, the CI system automatically runs thousands of regression tests. If a new change breaks an old test, the PR is blocked from merging.

18. Release Testing

Testing a release candidate intended for use outside the development team.

• Primary goal: convince supplier the system is good enough for use
• Demonstrates: specified functionality, performance, and dependability
• Demonstrates: system does not fail during normal use
• Usually black-box testing — test cases derived only from system specification

Release Testing vs. System Testing

19. Requirements-Based Testing

Examine each requirement and develop test cases for it.

Example — Mentcare System (Allergy Warning):

Scenario testing (Mentcare's George the Nurse) tests these features:

• Authentication (login)
• Downloading/uploading patient records to laptop
• Home visit scheduling
• Encryption/decryption on mobile device
• Record retrieval and modification
• Drug database integration (side effects)
• Call prompting system

🍎 Apple Health example: Each "capability" (heart rate alert, cycle tracking, crash detection) needs scenario-based tests using realistic user journeys — not just unit tests.

20. Performance Testing

Tests emergent properties — performance and reliability under load.

• Tests should reflect realistic usage profile
• Plan a series of tests where load is steadily increased until performance becomes unacceptable

Stress Testing

Deliberately overloads the system to test its failure behavior — what happens when it breaks?

🌉 Bridge analogy: Gradually adding weight until the bridge starts to bend — you want to know the limit before real traffic uses it.

🍎 Apple example: Before an Apple Event, Apple stress-tests the Apple Store app with simulated millions of simultaneous users hitting "Buy" at the same second. They need to know the failure point.

21. User Testing

Users/customers provide input and advice — essential even after complete system and release testing.

Why? — Influences from the user's working environment affect reliability, performance, usability, and robustness in ways that cannot be replicated in a lab.

Three Types of User Testing

🍎 Apple analogy:

• Alpha = Apple employees using pre-release iPhone builds daily (Seed program internally)
• Beta = iOS Public Beta — thousands of users finding real-world issues
• UAT = Enterprise customers validating a custom MDM solution before rolling it out to 50,000 employees

Acceptance Testing Process (6 Steps)

1. Define acceptance criteria
2. Plan acceptance testing
3. Derive acceptance tests
4. Run acceptance tests
5. Negotiate test results
6. Accept or reject the system

Agile + Acceptance Testing

• The user/customer is embedded in the development team
• Tests defined by user, integrated with automated tests
• No separate acceptance testing process
• ⚠️ Main risk: the embedded user may not represent all stakeholders

22. Big Picture Summary

TESTING FUNDAMENTALS
  Testing shows PRESENCE of errors — NEVER proves their absence
  Two goals: Validation (prove it works) + Defect Testing (break it)
  V&V: Verification ("built it right?") vs. Validation ("built the right thing?")
  Inspections (static) + Testing (dynamic) = complementary — use BOTH

THE 3 STAGES
  Development Testing → Release Testing → User Testing
  (dev team)            (separate team)   (actual users)

DEVELOPMENT TESTING (3 levels, bottom-up)
  Unit Testing → Component Testing → System Testing
  (isolation)    (interfaces)        (interactions/emergent)

KEY STRATEGIES
  Partition Testing — equivalence partitions, test one per group
  Guideline-Based  — based on experience of common errors
  Use-Case Testing — derive system tests from use case diagrams

TDD (Test-Driven Development)
  Write test FIRST → fail → implement → pass → repeat
  Benefits: Code coverage, Regression suite, Easy debugging, Living docs

RELEASE TESTING
  Black-box, separate team, validation focus
  Includes: Requirements-based testing, Performance testing, Stress testing

USER TESTING
  Alpha (dev site) → Beta (user's environment) → UAT (accept/reject)

🍎 Final Apple thought: "We run tests on tests." Apple uses XCTest for unit tests, UI tests for integration, and TestFlight for beta — three levels of the testing pyramid in production. But even with all that, every major iOS release has a public beta phase because testing cannot prove zero defects. That's why the last thing Donald Norman said about testing is the most important: you'll always remember the one time it failed.

Cheat sheet for CSS323 Software Engineering — Chapter 8: Software Testing You can't prove software is bug-free. You can only prove it has bugs. 🐛