Patch latching of Testing Challenge 1.2

You might have read the previous post. If not, it is here. It can help better to know the context of latch work that I'm about to do now. As per the author of this post, there is a major bug underlying in this work. As the word reads, "There was a major bug in Toughest Testing Challenge 1.2" – it also imparts meaning of just one bug i.e. major bug. I cannot take that word, as the author hasn't provided any ground work for knowing that number or how he derived at that figure.

Trying to know what that one major bug is, will get a smell of an unpleasing emotion triggered by pop up dialogs and/or its contents in web application. How did I know that was web application? By reading the URL in dialog title bar and touring that website. Doing so it asks me another question, "that dialog is used as an example to show technical writing stuff for ease of understanding the action or response; where's the problem which Dhanasekar is talking about in the dialog module?" This makes the exercise interesting for me to work on it.

Catching the patch

Now the management has come with idea to give this task for two vendors. Vendor one is asked to come up with test ideas which "*can* be executed in 15 minutes and time is nonnegotiable in this context. Vendor two is asked to estimate time required for testing the dialog module and then generate test ideas for the time estimated. The phrase "generate test ideas for the time estimated" is ambiguous one here. Does it say to arrive at ideas how the time is estimated or to come up with test ideas to test dialog in estimated time? Not sure; I go with second thought now – to come up with test ideas to test dialog in estimated time. I will not work on how I estimated the time required instead I will work on test strategy alone.

Assuming that client has communicated to Vendor One and Vendor Two what the problem is, can start of the exercise. What if the problem is not communicated and asked for the testing service?

• In either case, I would like to transpect the client asking her or his time for 3 to 5 minutes in knowing what makes that application more interesting and competitive from business view. Not sure can I get information what I'm looking in those 3 to 5 minutes. Besides I keep reading the client, so I can get the trust of her/him and ask what makes the person to feel concerned about the dialog module and collect few words of it. If I can get few words on these, it helps me. Had you noticed my strategy for testing is already started trying to talk with client in knowing what is bothering them. If client does not give time to talk, we both cannot help each other to the best whatever it is possible.
• Know what the client understands for 'dialogue'? Is it same as 'dialog' box I understand? Do we both have same understanding of this?
• What types of dialog box or windows exists in application?
• The problem described in dialog module is actually from the dialog implementation? Or the problem of else where is being observed in dialog(s)? How will I know this? What is causing the client to be so bothering with dialog?

Before going beyond from here, I will let you know what I understood for word 'test strategy'. I practice it as collection of ideas helping me in building the tests needed. My test strategy will connect the tests I perform and the purpose of the testing carried out.

Latching the patch

Context of tester at vendor one:

A client has approached tester for test ideas that can be executed within 15 minutes of time. Assumptions made here are – client wants immediate service; tester has no time to detail out the service request; and, tester is not aware of what the application is.

Who will be executing test ideas in 15 minutes which I give? Is it me? Or other person and will I part of it? Nothing is specified. This looks like more of imaginary request from client. It cannot be neat job for the context what the customer is in now though if it is delivered as a quick job. Hence I would keep off from testing dialog module in this context. Being a tester I want to be useful for those who hire me and educate them, when it is needed.

If stakeholder(s) wishes to speak further on this job, I will provide free consultation of 15 hours in identifying potential problem(s) in dialog module. Testing is not 'about convincing' or 'to convince'. It is with providing information which critically matters to stakeholder's who have vested interests and seeking testing.

Context of tester at vendor two:

A client has approached tester for a service asking estimating for testing the dialog module. Also to write test ideas for testing in the same estimated time provided by vendor two. No mention of time restrictions explicitly made to tester. From this I can also understand that tester has the time by which the testing has to be stopped.

Strategy of vendor two:

• Build the mental model of what will be tested. Sketch it out and use by updating it consistently. This to be in simple representation so that a person who is related to application being tested will understand it with minimal words.
• As said will talk to client about what's the concern in dialog. Know about the most important part of application per business needs. Know the potential users of the applications.
• Work to know whether the concern with dialog is functional, UI, UX, hardware, software. This can tell me where should my tests focus and look out probable risks and problems.
• Use FAILURE heuristic of Ben Simo which may be handy tool in this context.
• Try to collect all programmed messages or contents displayed by application along with context. Trigger the condition such that all programmed messages are shown in the dialog. Assess the problem in dialog by this action and extend it beyond to find impact from this behavior if any.
• Note if any messages that are not programmed and still seen in form of a dialog box in application. Know its context and evaluate the potential problem. Look what can be done to avoid if this behavior is pestering user.
• Collect any specific data or probable data which pops up dialog on screen while using this application. Use such test data in testing and evaluate the context to know the problems from dialog.
• What types of dialog boxes exist in the application? Work with programming team or technical person who knows about this.
• Look for any prototypes available for dialogs available in application. Think of problems that may arise from it.
• What kind of controls exists on dialog box, if any? Does it make sense to user for having it on dialog though it appears as per need and functioning?
• Do the buttons or GUI controls are relevant about the actions it does or information it provides? Ex – naming pattern, display of control names.
• Does the sentence or phrase or text tell importance of dialog box shown in simple words? Is this understandable by any users who use application?
• Is the user is educated what to do when not sure what to do seeing a dialog box while using the application? As application cannot be tweaked to comfort for each user who purchased a license or using it, educating user can help.
• Any option exists to stop or avoid or show or to show later, the dialog which user does not want to see?
• Enabling and disabling of GUI control should be contextual and can it be understood by the user why it is so then?
• If any default values or options available in dialog and it is selected. Will it create problem to user or business in later stages? Any intimation about this while selecting such options?
• Content area and spacing of dialog box to be appropriate for various resolutions, screen width and devices.
• Collect the count of dialog that user will have to go through for validation of entry made in a web page.
• What kind of information shown on dialog box? Can user follow it? Is it interactive enough to fulfill purpose of dialog box? Will it communicate as it is talking to me?
• Do dialog helps me take decisions with affirmative questions and/or words and GUI controls?
• Does dialog structure have all its details? Ex: Title, type of message or dialog box, action, GUI controls etc.
• Can dialog box be minimized, maximized, re-sized? Is it modal in a context where it requires being modal dialog? Is it a non-modal in a context where there is no need for a modal dialog? Will it intimate the user visually when a non-modal dialog exists and trying to use application? If user returns after a while and have modal or non-modal dialog is it easy to identify that?
• How the dialog box is identified in online help or contextual help? Can user easily identify and know about this from Help or from customer support or by any other means that suits to context?
• Any picture or video or voice in dialog available on dialog? What does it tell to the user?
• Accessibility of dialog box. GUI controls in dialog and problems of having it, if any?
• Each GUI control has unique name so it helps in identification of it while debugging, investigating, and customer support helping the user?
• Which character is used as shortcut key? Is that character is visible enough to know that it is a short cut character for that particular GUI control?
• Does clicking on GUI control in dialog box take user to other place from current page? If yes, how does the user come back to the page which was being used? Will data entered be persisted?
• Is user intimated of navigating from current page on clicking any GUI control in dialog and its impact, if any?
• Indication of action initiated by using dialog box is success or failure?
• Is dialog shown for every single validation or user error? Is there any alternate for showing the message and reduce the mouse clicks or keyboard actions?
• Showing the related items in a dialog box according to context.
• Whether the repetitive controls and contents in dialog box is avoided or reduced in appearance?
• Look what actions are invoked using the GUI controls on button? Are those actions as expected?
• Look for the data transferred or persisted or modified or updated or left unmodified using a GUI control on dialog box. Any sensitive data or information is under threat in a context by creating problems with costs?
• What protocols are invoked using dialog in the application? What problems can it create here?
• How the post and partial post back happen using the GUI on dialog, if any such scenario in application?
• Can the actions of dialog box be tweaked from possible places where it can be done?
• Is it possible to bypass the dialog box operation which is mandatory in an application? If yes how? What are the problems doing so?
• If I have any other tester(s) worked/working on these will sit with together and brainstorm ideas how we FAIL if we did not meet purpose of this testing. Collect the ideas how to build our tests and prioritize those per information needed from testing by client.
• Collect the contexts where and when the dialog will be displayed in an application. Ex: OS, browser, screen resolution etc. With this I'm focusing on concern by learning application and its users.
• If accessible/allowed, will try to get the data from bug tracking system of client to know what kind of problem or behavior is described for dialog shown. What is the decision of management and technical team for such bug? Talk to customer support, sales and marketing team and go through reports of them for dialog problem written if any. Refer online or/and contextual help of dialog in application if any. Search on web and social media if any word has been put out on this application's dialog or similar application or other application.
• Recollect testing practice experience to see any such contexts. Make note and tweak it to the present context on finding any such work to present purpose of testing.
• If allowed use testing friends who have expertise in this kind of testing on obtaining permission from client. In case not allowed will talk to such friends providing generic example from any one open source application. Learn how they tackle such related problems with the questions I'm finding hard to find answer. Respect to NDA is maintained by adhering to it.
• Gather information from test regarding whether dialog shown from application being tested or any other application interacting with testing application. Identify scenarios where the dialog and content in it matters most. And know who are the potential users of this application and think of what would they do. This will ensure I'm practical in terms of executing the tests.
• Keep reviewing and updating the strategy to the context so that it benefits the client and tester.

Will explain this strategy to client who wants me to execute the test ideas in the time I will have for testing.

Estimate: I test till the stakeholder is less interested in information that she or he is looking from testing and asks me to stop testing be it for any reason, or I break the contract of job. If I'm not involved in testing with these little ideas, need to talk with client to know more about the how the contract of work will be.