In one of the recent discussion with fellow testers, the topic was "testability". While fellow testers asked, "what is testability?", I had a question which I shared in the discussion -- "What is codability?".
Codability and Testability
I see both of these are inter related. If it is codable then it is testable to a degree. What is that degree? That's the question of interest. But, what is that one will accomplish by knowing the order of degree, here? This forms the base in having the curiosity or wish to know about testability.
I have a product's feature which I have to implement. The feature has cases which I need to handle in run time for different situations which a user can encounter. Well, I can code for situations which I can think and which is of priority from point view of using the product. Then for other situations which I'm not seeing or not thought off, will it be a hit for my product? What is the complexity level of the code I'm writing for this feature and what are the risks that are subtle in code I have written? The written code will always have the risk that is tied along with it.
Now, can I tell that testability is influenced by multiple factors and not just on the basis of the test identified? Well, after knowing this, it is essential to understand -- "testability is the easiness at which I can test the system in a given situation.".
Factors influencing testability
The few factors which will potentially influence the order of easiness -- in learning the test challenge, identifying the test, approaching the test and executing it, are as below:
- The testing skills of a tester
- The programming skills of a tester
- The technology skills of a tester
- The experience of a tester in testing such systems
- What is know by tester about the product -- purpose of product; people who are building the product; people who will use or who are using the product; the features and functionality of product; technology of the product; code written for the product; how the product is built with limitations from business team; information available about the product; process, practice and culture followed by teams in building product; availability of the system and its present state
- Time available for system development
- Time available for testing
- The level or order to which the testability aid is built and can be provided in system to test, and the complexity or simplicity level of same
- Measures taken to improvise and include testability aiding stuffs in system
- Existing tools or utilities for testing the product or code, or to build one such utilities
- The freedom to tester and testing to control and guide the testing activity
- Information expected out of testing and priority of it
- Relationship with fellow testers, fellow programmers, business teams and any other who are involved and interested in the system being programmed and tested
- The audience interested in test report or outcome of testing and their expectation
Testability and Test Coverage
The test coverage accomplished by a tester in given context is relational to the skills set of tester and the testability factors of the system and environment.
While I have understood this, I tried to learn the scale of testability keeping the test coverage as base. Here is what I have observed for now.
By keeping the test coverage as a base scale might mislead me to learn what I can accomplish in coverage. Because, the testability might be available and easy one for me to accomplish the 10th level of coverage with my skills and other influencing factors of testability. But to cover and accomplish the first three levels can be not the easy one given my skills and other influencing factors of testability. The vice versa is also possible. Here, I can take longer time to learn and know how could I have cut short and still accomplish the same coverage given the testability level to me in a context, if I had learned about the complexity level of of it. While I reach to a intended level of coverage, I would have done the moves which does not favor business timeline, learned from them and repeated few stuffs. This is not wrong but I have consumed time which I could have invested in testing to accomplish much more coverage. The same is represented in below image.
Keeping the test coverage in base and assessing the testability on vertical scale marking the breadth and width of coverage spread out in parallel which I want accomplish or what I can accomplish, here is what I observe for now:
Understanding the test coverage I want to reach by my testing and automation, so I provide the information expected is a challenge each time. The reason for simple to quote for first is the 'testability' for that coverage mark. Once I get an idea of what is the skills that I need to build and use in such situations, it will help me to make strategic decisions in testing execution and its management. The is very important as per me from point of view testing strategic base. This is helping me and allows me to test the perceived level of testability itself for marked coverage boundary or milestones.
This base of strategic thinking in learning and identifying the testability can be used in timelines of a project. Not just during the execution of testing, also in pre-execution and post-execution, it can be used. It gives an idea of how the tester has to be equipped in changing needs of testing and engineering. For me this is working in context where I'm exposed and in few cases I had to add few vectors along with test coverage such as technology and programming factors specific. I'm experimenting this in varied projects having different technology and skill challenges.
Visibility out of testability
With this, for now I see,
- Testability is related with codability, programmer, tester, environment, process, people, situation, priorities
- Codability gives the first hint on testability and skills needed to test the same.
- Testability influences the Test Coverage
- Testability influences the time taken by a tester to test the system in a given context
- Like codability requires the skills of a programmer, testability requires skills of a tester
Note: Automation when leveraged with testability, wonders can be done via automation in assisting the testing.
