Adam Roman
Thinking-Driven Testing. The Most Reasonable Apprach to Quality Control
Springer International Publishing AG, 2018
ISBN 978-3-319-73194-0
ISBN 978-3-319-73195-7 (eBook)
DOI: https://doi.org/10.1007/978-3-319-73195-7
305 pages, 92 figures, 30 tables
This book presents a new paradigm of software testing by emphasizing the role of critical thinking, system thinking and rationality as the most important skills for the tester. It thus approaches software testing from a different perspective than in past literature, as the vast majority of books describe testing in the context of specific tools, automation, documentation, particular test design techniques or test management. In addition, the book proposes a novel meta-approach for designing effective test strategies, which is based on recent advances in psychology, economics, system sciences and logic.
Chapter 1 starts by introducing the fundamental ideas underlying software testing. Chapter 2 then describes meta-strategies in software testing, i.e. general approaches that can be adapted to many different situations that a software tester encounters. Next, Chapter 3 presents the concept of Thinking-Driven Testing (TDT). This approach utilizes the concepts discussed in the two previous chapters and introduces the main ideas that underlie a reasonable and optimal approach to software testing. Chapter 4 builds on this basis and proposes a specific approach to testing, called TQED, that makes it possible to increase creativity in the context of delivering effective, optimal test ideas. Chapter 5 provides an overview of different types of testing techniques in order to understand the fundamental concepts of test design, while Chapter 6 details various pitfalls a tester may encounter and that can originate from a wide range of testing process areas. Lastly, Chapter 7 puts all this into practice, as it contains several exercises that will help testers develop a number of crucial skills: logical thinking and reasoning, thinking out of the box, creativity, counting and estimating, and analytical thinking.
1 Introduction
1.1 Why this book?
1.2 What is this book not about?
1.3 What is this book about?
2 Fundamentals of Software Testing
2.1 Introduction
2.2 The beginning: what is testing really about and why is it so difficult?
2.2.1 Testing as a unique activity. Why testing is so hard?
2.2.2 Testing finds bugs, but this is just the tip of the iceberg
2.2.3 Quality control vs. quality assurance
2.2.4 It’s not about the working code, but a solution to the business problem
2.2.5 Your real boss is your client
2.2.6 Continuous scale of tests: from confirmation to ‘dirty’ testing
2.2.7 Test redundancy is not always bad
2.2.8 Disregarding risk is risky
2.2.9 Feedback for the team
2.2.10 It is not about test execution
2.3 The mythical test-case
2.3.1 A primary function of test cases
2.3.2 When should we use test cases?
2.3.3 Benefits from test cases
2.4 The nature of defects and the myths about them
2.4.1 Defect distribution in time
2.4.2 Cost of defect removal
2.4.3 LOC metric as a defect predictor
2.4.4 Defect grouping
2.4.5 Defects resulting from interaction of parameters
2.4.6 Module structure vs. defect-proneness
2.5 Exploratory vs. scripted – an apparent problem
2.6 The ideal tester’s skillset
2.7 It’s all about communication
2.8 Testing process – pure abstraction or tangible reality?
2.9 Models in testing
2.10 A bit of philosophy: problem of universals from the tester’s perspective
3 Testing strategies – how to become a better tester?
3.1 Introduction
3.2 Be inspired – what can a tester learn from...
3.2.1 … philosophy
3.2.2 … psychology
3.2.3 … economy & management
3.2.4 … mathematics & logic
3.2.5 … probability & statistics
3.2.6 … systems science
3.2.7 … ergonomics
3.3 The tester’s mind: psychology of testing
3.3.1 Creativity
3.3.2 Cognitive bias
3.3.3 Confirmation bias and Wason experiment – software testing as hypothesis testing
3.3.4 Wason selection test: why language is so important?
3.3.5 Cognitive dissonance
3.3.6 Conway’s law
3.3.7 Csíkszentmihályi’s Flow Model
3.4 Useful methodological laws and tools
3.4.1 Occam’s Razor
3.4.2 Descartes’ doubt as methodological skepticism
3.4.3 Optimization and prioritization: Pareto 80-20 principle
3.4.4 Time management: Parkinson’s Law
3.4.5 Burch’s four stages of competence
3.4.6 More eyeballs is better: Linus’s Law
3.4.7 Legacy code: Eagleson’s Law
3.4.8 How to hire new people: Peter Principle
3.4.9 Meeting facilitation: Sayre’s Law
3.4.10 Data centralization: Ellison’s Law of Data
3.4.11 False sense of security: Spafford’s Law of False Alerts
3.4.12 Beizer’s Pesticide Paradox
3.5 Standards and norms – best practices or pure evil?
3.6 Models of software errors and why we should use them
3.7 ‘Plans are useless, but planning is indispensible’
3.8 The eternal problem of documentation
3.9 Underestimated estimation, unpredicted prediction and measures not measured
3.10 Simulations – your manager loves them
4 Thinking Driven Testing: a universal testing framework
4.1 Introduction
4.2 TDT manifesto
4.3 Universal rules
4.3.1 Know your client’s needs
4.3.2 Know where you are
4.3.3 Know your enemies (faults)
4.3.4 Be reasonable and professional
4.3.5 Be optimal and always bring added value
4.3.6 Ask the right questions and don’t be afraid of asking
4.3.7 Don’t give up
4.3.8 Think
4.4 There’s no such thing as ‘automation tester’
4.5 Dimensions of testing and process measurement
4.6 Critical thinking
4.7 Systems thinking
4.7.1 Eleven laws of systems thinking
4.7.2 Systems thinking and software development models
4.8 Personal development in software testing
4.9 Design Thinking or Good, Ol’ Fashioned Requirements Analysis?
4.10 Why studying software testing?
5 TQED model
5.1 Introduction
5.2 Origin of the model
5.2.1 The background
5.2.2 Inspiration from physics
5.3 Model features
5.3.1 Simplicity
5.3.2 Understandability
5.3.3 Flexibility
5.3.4 Genericity
5.3.5 Educational value
5.4 Model structure and model components
5.5 Heuristics for model components
5.5.1 Data
5.5.2 Events
5.5.3 Quantity
5.5.4 Time
5.5.5 Combinations of dimensions
5.6 Using the TQED model for building an effective test approach
5.6.1 Formulate the test problem
5.6.2 Identify subproblems
5.6.3 Identify objects
5.6.4 Identify combinations
5.6.5 Identify test ideas
5.7 Model’s universality
5.7.1 TQED as generator of test approaches
5.7.2 TQED as a generator of formal test design techniques
5.8 TQED in practice: a ticket machine example
5.9 Conclusions
6 Testing techniques
6.1 Introduction
6.2 Test oracle problem
6.2.1 Test oracle problem from the theoretical point of view
6.2.2 Test oracle problem from the practical point of view
6.3 Testing based on document analysis
6.4 Testing based on a software model
6.4.1 Business software models
6.4.2 Model as a form of idealization
6.4.3 Technical software models
6.5 Testing based on code
6.5.1 Code review
6.5.2 Code as a mean for coverage metrics
6.6 Testing based on error model
6.7 Testing based on quality model
6.7.1 McCall’s Model
6.7.2 Boehm’s Quality Model
6.7.3 ISO 9126 and ISO 25000 Quality Models
6.7.4 Problems with quality models
6.8 Testing based on randomness and statistics
6.9 Testing based on intuition and experience
6.10 Testing based on other people
6.10.1 Who?
6.10.2 What?
6.10.3 When?
7 Testing pitfalls and how to avoid them
7.1 Introduction
7.2 Deceptive coverage metrics
7.3 Reports and data presentation
7.4 In models we trust. Or do we?
7.5 Selecting the technique
7.6 Variance and its interpretation
7.7 Bad planning
7.8 Hastiness
7.9 Using the tools: you’re doing it wrong!
8 Kata and exercises for testers
8.1 Introduction
8.2 What is kata?
8.3 How to practice the tester’s kata and exercises?
8.4 Kata and exercises on logical thinking and reasoning
8.4.1 Kata #1: logical analysis.
8.4.2 Kata #2: what-if analysis.
8.4.3 Exercise: logical thinking
8.4.4 Exercise: who lies?
8.4.5 Exercise: Bob’s daughters
8.4.6 Exercise: mismatched figure
8.5 Kata and exercises on creativity and out-of-the-box thinking
8.5.1 Kata #3: generation of ideas.
8.5.2 Kata #4: bypasses and detours
8.5.3 Exercise: the right question
8.5.4 Exercise: connecting the dots
8.5.5 Exercise: find the rule
8.5.6 Exercise: four triangles
8.6 Kata and exercises on counting and estimating
8.6.1 Kata #5: estimations for counting.
8.6.2 Kata #6: time estimations.
8.6.3 Exercise: Monty Hall problem
8.6.4 Exercise: probability
8.6.5 Exercise: simple counting
8.7 Kata and exercises on analytical thinking
8.7.1 Kata #7: risk analysis
8.7.2 Kata #8: abstract testing
8.7.3 Exercise: testing a chair
8.7.4 Exercise: the fly and the trains
8.7.5 Exercise: code analysis
8.8 Solutions to exercises
Appendix A. Glossary of the most important terms
Appendix B. Testing in a nutshell
Bibliography