Objectives
How can we meet the Objective? (Goals)
How can we meet the Objective? (Goals)
Tests Should Help Us Understand the System
Tests can show us how the code is supposed to work
Tests as Documentation - automated tests act as a form of documentation
Write Tests First (TDD)
tests save us a lot of debugging effort
Writing the tests before we write the code forces the code to be designed for testability
Hard-to-Test Code
symptoms - hard to test code
impact - we cannot easily verify the quality of the code in an automated way
causes
Highly Coupled Code,
Asynchronous Code,
Untestable Test Code
Tests Should Require Minimal Maintenance as the System Evolves Around Them
Robust Test - write tests in a way that the number of tests impacted by code changes is small
Communicate Intent
Intent-Revealing Names
should be apparent which inputs result in which outputs
Fragile Test
symptoms - frequently, tests that used to pass are now failing after a change
impact - increases the cost of test maintenance
causes -
Interface Sensitivity,
Behavior Sensitivity,
Data Sensitivity,
Context Sensitivity,
Overspecified Software,
Sensitive Equality,
Fragile Fixture
Tests Should Reduce (and Not Introduce) Risk
Safety Net - tests allow us to make changes more safely, can take chances on different code changes
Do No Harm - the tests should not introduce new kinds of problems
Verify One Condition Per Test
makes Defect Localization easier
prevents missing a test when an assertion fails
does not have to mean one assertion
Conditional Test Logic
symptom - may not produce symptoms, but it should be suspicious to the reader
impact - difficult to know what a test is going to do, how do we know that it will actually detect the bugs it is intended to catch
causes
Flexible Test,
Conditional Verification Logic,
Production Logic in Test,
Complex Teardown,
Multiple Test Conditions
Tests Should Be Easy to Run
Fully Automated Test - tests that can be run without manual intervention
Self-Checking Test - the test verifies the outcome and communications pass/fail
*Repeatable Test - the test can be run multiple times and get the same result, without intervention
Keep Tests Independent
An Independent Test can be run by itself
Interacting Tests or Erratic tests tend to fail in a group and can be harder to debug
Independent Tests give better Defect Localization
Erratic Test
symptom - one or more tests giving different results depending on when or who is running them
impact - tempted to remove it (Lost Test), may obscure other problems
causes -
Interacting Tests,
Interacting Test Suites,
Lonely Test,
Resource Leakage,
Resource Optimism,
Unrepeatable Test,
Test Run War,
Nondeterministic Test
Tests Should Help Us Improve Quality
Tests as Specification - tests enable us to specify the behavior in various scenarios captured in a form we can execute
Bug Repellent - tests keep bugs from crawling back into our code
Defect Localization - if our tests are small, we can pinpoint bugs pretty fast with tests based on which test failed
Keep Test Logic Out of Production Code
production code should not contain any conditional statements of the "if testing then" sort that is running alternative logic or preventing certain logic from running
it should not contain any test logic
if the system behaves differently under test, how can we be certain the production code actually works?
test hooks could cause software to fail in production!
Developers Not Writing Tests
symptom - we hear they aren't or have observed production bugs
impact - creates tech debt, will take longer and longer to add new functionality
causes
Not Enough Time,
Hard-to-Test Code,
Wrong Test Automation Strategy
Tests Should Be Easy to Write and Maintain
Simple Test - tests should be small and test one thing at a time
Expressive Tests - build up a library of domain-specific testing language
Separation of Concerns - keep test code separate from production code
Ensure Commensurate Effort and Responsibility
the amount of effort to write or modify tests should not exceed the effort to implement the corresponding functionality
the tools required to write or maintain the test should require no more expertise than the tools used to implement the functionality
Obscure Test
symptom - having trouble understanding what behavior a test is verifying
impact - makes tests harder to understand and therefore maintain
causes
Eager Test,
Mystery Guest,
General Fixture,
Irrelevant Information,
Hard-Coded Test Data,
Indirect Testing