Tuesday, December 3, 2019

Test Charter and the Four Components of a Charter



A tweet from Ministry of Testing was -- Do you use Test Charters?

Speaking out of my so far practice, I heard the word 'charter' 10 years back when I was referring to SBTM in James Bach website.  Till then I had not heard the word -- charter.  The two words which confused me then was -- Mission and Charter.  

I looked into the sample SBTM reports which James Bach had shared.  I observed the word 'mission' for each session.  That is each session is chartered with a mission to accomplish.  Meanwhile, I strongly believe the spark that took off to testing globe wide referring such testing resources had its birth in Bengaluru then.  Most will say nah and disagree to it.  But that's the truth and Weekend Testing was one of its outcome.

Coming back to the subject of charter, I say, it was not that easy then for me to write one.  I got better each day and I try to get better today as well.  While I worked in the role of Test Coach in Moolya Software Testing, I pair tested with freshers and lateral hires.  I insisted the testers (whom I was assisting to practice testing) to write charter on their own.  I could see the difficulty they were going through and it was a mirror of me showing what I went through some time back.  In beginning all had difficulty to practice in session as they could not concentrate for 30 minutes (forget 45 or 60 min or 90 min) and to take make test notes in parallel.  Yet, there was no practice without a session and a charter to it.

To keep it simple and make lives of the tester easier, I broke the charter into four components. They are:
1. Intent  -- What is the intent of my testing in this session?
2. Target -- What I should be testing?
3. Resources -- What I should be referring while I test?
4. Information -- What information I have to learn from the tests and how to report it.

For example, say the charter of a session is -- Browse through the different doctors displayed based on specialty by swiping the doctor info card in the app. Validate if the subsequent result pages of doctors displayed is still relevant to the specialty chosen, while a doctor can have multiple specialty.  Refer to the HTTP request and response coming in for each result page and validate data displayed on client end. Report with logs and test data when you find or feel there is a problem.

The same charter can be written as this -- Test for the specialty displayed for doctor in search result is consistent in each result pages returned by server. Make sure the test data of doctor have multiple specialty. Report with logs and test data when you find or feel there is a problem. Make note of the confusion if you have any in analyzing the search result.

Also it can be written as this -- Test for the doctor card shown in search results based on specialty chosen.  Report with logs and test data when you find or feel there is a problem.  Make not of the confusion if you have any in analyzing the search result.

If I had to break this charter into four components, then:
1. Intent -- Search for doctors based on a specialty
2. Target -- Doctors info card displayed in the app that is search result pages
3. Resources -- Specialty of a doctor; HTTP request and response; Data displayed on client app
4. Information -- The search results returned and displayed are consistent with chosen specialty? Report if any problem and confusion.

All the above written three charters mean the same.  But how it is written and details provided, varies.  What style to pick and details to provide it depends on multiple reasons when I work with team.  When I work as a only tester testing, I keep the charter pretty self explanatory with details to the reader of session notes.

In my opinion, a tester gets better in writing charter when these four components is clear and can be identified. 


Backward Compatibility - What and who drives the compatibility?



A tweet of Ministry of Testing read as this -- How do you decide how backwards compatible your product should be?

I learn, there is no general rule and as well no specific rule that applies when it comes to backwards compatibility, forward compatibility and the compatibility.  It is highly specific to what product we are dealing with and how the system around it are developed, or being developed or will be developed.  Also how it is deployed.

While compatibility is one of the main feature of product feature, it also signifies -- the design, the deployment style & patterns, communication and the exchange of data.

Further to be precise, compatibility is context specific task.  I have not come across where it is generic task where it is applied and applicable to any where.


If asked what software can come into compatibility spectrum, then:

>> Database System
>> Server
>> Client libraries like SDKs
>> Server Configurations
>> Operating System
>> Hardware Configurations
>> Client application versions being used by user and number of users
>> Server configuration versions being used in different client applications
>> Browser versions, it was a major compatibility concern a decade back
>> Application system version and its dependencies compatibility version


Few analytics that influences the decision on compatibility

>> Analytics showing the user on specific versions of libraries
>> Analytics showing the user on specific versions of OS
>> Analytics showing the user on specific version of hardware configurations
>> Analytics showing the user using specific feature which involves server configurations
>> Analytics showing the conversion funnel on specific setup of system


Few emotion that influences the decision on compatibility

>> Experience of user using a product on specific setup and configuration
>> Customer Support analytics on problems, queries and user satisfaction on specific setup and configuration
>> History of previous compatibility and migration
>> Competitor's presence
>> Team's opinion


Business aspects

>> Negotiation extent
>> Bearable Cost and Non-bearable cost
>> Business Value
>> Social Value
>> Strategic Value
>> All above converting to business profits
>> Survey carried out
>> Timeline
>> Budget and Resources
>> Talents -- people


Environment

>> That simulates the backward version of the system
>> That simulates the provisioned version of the system
>> Outcome of the assessment in such environments


If seen, where do tester fit here to decide the backward compatibility of the product?  In my experience, the decision of backward compatibility is most times strategic from aspect of the technicality.  In the enterprise ecosystem, the same strategic will weigh more on the business decisions and then the technical decisions.

The executing tester most time will not be involved here in such decisions.  Could be a person leading the testing team or the engineering efforts can be involved in such meetings.  In start-up environment I have worked so far, the decision meeting most times did not have the testers involved.  However, on getting the heads up for backward compatibility need, I use to assess the risks in system and its sub-systems.  Then, I shared and had a meeting scheduled if there was a need of different stakeholders and teams to clarify and resolve any risks mentioned.

In simple, the business don't want the service to be broken for a user.  The user should get the service that business provides on using the software application.  Further, the conversion what the business expects has to be what the business expects.  If not, then there is a problem to user and as well to business.