Time to Update the 101s for Test Engineers

 

The thought I'm sharing in this post is growing stronger in me, in recent years. I shared it as points with the community in the #21Days21Tips initiative from The Test Chat.  I post it on my blog as well so that it will be a reminder to me when I scroll through my posts.

The 101s for Test Engineers

Today, the first few stepping lessons of Testing and Automation to be on -- "How to Overcome the Fear of Failure".  If this is not available in the team and organization, it is time to start and build it.  The present and future Test Engineers need it.

The starting lessons or 101 of the Testing and Automation need not necessarily be:

• What are Testing and Automation?
• Test Design and Techniques
• Programming Language
• Tech Stack, Full Stack
• Clean Code
• Design Patterns
• Tools
• What should I pick; how & where to start here?

All these will connect and align when one can handle self-doubts and fear of failure.

Approach Goals and Avoidance Goals

As a team, organization and mentor, help your Test Engineers to have "Approach Goals" than letting them build "Avoidance Goals" out of fear.  I see practicing the Approach Goals is a life skill, today.

I take the below lines from this post:

Approach Goals are those with positive outcomes that we work towards.  Positive can mean different things in different contexts, such as liked, desirable, pleasurable, or beneficial.

Avoidance Goals are those with negative outcomes that we work to avoid. Negative can mean different things, including disliked, undesirable, painful, or harmful.

Examples:

• Approach Goals
• Know why I have those fears when I think of automation?
• I need to be skilled in automation. How can I manage my fears and proceed with practicing automation?
• I need to do better testing with help of automation. What I should practice for doing this? What help I will need?

• Avoidance Goals
• I will start practicing automation next month
• I will start automation once I understand and write good code
• I will start automation once I understand Design Patterns and Clean Code
• I will start automation once I learn how to write a framework
• I need to do it well; let me learn the basics very well first and then let me do automation
• I will check for workshops and people who can help me to do automation

Approach Goal: I will start it! No matter how small it is, I will start. I will keep updating as I learn. I start it from here -- Automate a browser to launch and open a new tab.

Frame the Approach Goal that helps to take one ahead by dealing with the fear.  The goal needs to make sure the milestones of learning are highlighted to one.  That way, one feels confident and has clarity along with skills of learning better be it what is new or what one is aware of.

Note: If the question is why I talk of automation in the examples of Approach Goals and Avoidance Goals, this is what we hear from the majority of Test Engineers today.

My Experience

When the Test Engineers know to build and use the Approach Goals, they will align and practice better on subjects:

• With which Test Engineers are unfamiliar
• With which Test Engineers are uncomfortable

Also, help the Test Engineers to build skills to spot Avoidance Goals.  So that, they are aware of the Avoidance Goals they tend to have and construct positive Approach Goals to learn new skills and solve the problems.

Understanding - Are Framework and Library the Same?

We Have This Question

• What is a Framework?
• What is a Library?  

I had these questions when I started my testing career.  At times, I ask this question today as well, when I listen to fellow engineers talk about it.  

I can say firmly that you had these questions.  You would have asked these questions to yourself or to your fellow testers.

Using Interchangeably

In the teams in which I have worked and in the community, I have seen these two terms are used interchangeably.  This has added to the confusion.  At times you might see the programmers using these two words interchangeably; but, they might know what it does and how to use it.

If I were to start my automation practice, to what should I call framework and library?  Does this question have crossed your mind and thoughts?

Framework and Library - What it is?

I understand both Framework and Library as code.  Yes, both are programming code that is structured and organized in a way.  This code is written by a programmer and will be consumed by other programmers and testers.  

Why do we use it?  We use it to assist in solving a problem we have in our work.  May be is what leads one to get confused or use these words interchangeably.  But are they one and the same?  I had this question.  I spoke to programmers and testers with whom I worked.  I observe each of them having their own ideas and perspectives about it.  

But are they one and the same?  As of today, I understand Framework and Library are not the same. It differs in -- the ease of consumption as is and the code to be written and organized for using it.  I see these two factors differentiate Framework and Library; there could be more factors and I'm not aware of it.

Framework and Library - My Experiences

I use libraries in testing and automation I do.  For example, 

• the Selenium library to interact with the browser and perform the actions on the web pages
• the RestAssured library to build the HTTP request
• the Appium library to interact with the mobile OS and its app's UI

I shared the above examples as most of us are aware of them and relate to them in our work.  Then, what is the framework?  

Have we heard of these words -- React, Angular, Bootstrap, and Flutter?  Today we use them to build web apps and mobile apps (using Flutter).  These are frameworks consumed by programmers to build the applications.  Some of these frameworks provide libraries to test as well.  For example, Chai and Jest assertion libraries.

Trying to understand further, one of my programming friends said me about the principle -- Inversion of Control.  Inversion of Control allows a framework to inject dependencies, inject configuration and manage lifecycle. That way programmer's program can focus on what it must do and not have to worry about any additional responsibilities.

I learn, Inversion of Control refers to transferring the control of objects and their dependencies from the main program to a container or framework.  How this will be implemented depends on the programmer and the design used to implement the same.

Note: In one of the perspectives I learn the framework provides the abstraction to its libraries.  We use these abstractions of a framework and develop our applications.  Can I say framework means abstraction?

Difference: Framework and Library

In the way we communicate, most times it is communicated as there is no difference between them and both mean the same.  Probably this is the reason we have the confusion, especially the Test Engineers who want to start the practice of automation will have this confusion.  This is being passed on to the next generation of Test Engineers who come take up Software Testing as a career.

Technically, the difference exists between the Framework and Library.  One of the key differences lies in the principle mentioned above -- Inversion of Control.  Let me walk through this with a use case that we Test Engineers cross every day.

Use Case in using a Library: Automating a user action on a web page

• I make use of the Selenium library
• I initialize the WebDriver and a driver of a browser
• I open a browser on my machine using WebDriver communicating with the browser's driver
• I talk to the browser to launch a URL
• I make actions on the web page

I'm not writing a code to do all of the above said.  I'm making use of APIs exposed by the Selenium library to do all of these actions.  The library's code handles all these for me.  My code will be for -- to consume Selenium's API and to aid evaluation of business logic -- the Utils is a commonly used word to describe the Class which has methods for the same.  

When I'm using a library, I have control over the code that I write in automation in deciding how the automation should execute that is in what order and what flow.  I can decide when to make what actions on the web page here.  The Selenium library does not mandate it on me.  

The only mandate I see is the sequence of actions that has to be made when using a particular API.  For example, the action events to be made in what order when used the Touch Actions API.

Use Case in using a Framework: Building a web and mobile app

• The team developing Android and iOS mobile apps using Flutter
• The team developing a web using Flutter or React

Just by using the Flutter and React frameworks, the apps and website will not be ready.  The programmer follows the structure and sequence insisted by the framework in her/his code.  The framework will tell how the code should be written and probably this also means the flow of code is controlled by Framework.  For example, when loading the content on the web page and screen.

In one of my current projects, the place where the message is displayed, how it is displayed, and how to interact with it, is a flow in code.  The programmer has to work oblige how the framework parses the error message.  It has to be done that way unless the programmer writes a custom code and plugs it into the framework, provided the framework supports it.  

This code which the programmer plugs in now, it is a custom library developed to use with a being used framework.  Now using this custom-built library, the reading or/and display of error messages can be made as per the need of the product.

Automation: Do We Write Library or Framework?

When we testers say automation framework, I see that we are referring to the structured library we have written consuming a framework and library.  For example, Arjuna developed by Rahul Verma is a framework.  I will consume Arjuna and build a library that has tests to automate web applications using the library as Selenium.

The automation I write consuming a framework and/or a library will have methods. Few of these methods will be written once and can be used in multiple places.  That is no repeating code and practicing the reuse of code.  It is like DRY -- Do Not Repeat Yourself.  We can see this practice in the automation we write.

For example, the below code can be written once in a class and invoked in different places where we test the business logic.  Now, these two methods will act as a library code and it is also part of a library code.  

We just call the method name and pass the parameters in different classes where it is needed.  No need to implement these methods each time in business logic where it is needed.  

Doesn't this give a perspective that a framework includes a library within it?  And, this questions me --  should I call automation what I do as an Automation Framework?  I say NO!  Unless someone uses my work as a foundation to build their tests and it dictates how the code should be written and structured, I do not call it a framework.  

But, one can use the classes that I write in their work.  This class act as a library.  Recall the class file name with Utils; this file has methods that we use by invoking it across the package.  One good example of this is, the logging part we write in our automation.  This logging part of code can be built as a library and can be used in different automation projects in which we work.

This is my understanding of Library and Framework at this point in time.  You have any questions or other perspectives that I'm not seeing, do share.

Solving: Question on Automating a Finance Web Application

 

Requirement for Web App Test Automation Framework

I read this question on The Test Tribe's Discord community server.

Here is the copy-paste of the question:

Hi all,

Could I have your expertise on below.

I have a requirement to build a test automation framework for a Web Application. Web application is developed using java and MySQL.

And the domain is related with finance sector. 

Most of the functions in the application is working with scheduling services which the service will triggered automatically. Mainly they wanted to verify the data which are displayed in UI with the database. Which means the data which is created with the scheduling services are added correctly. 

Some of the testing scenarios are as below:

1. For each transaction it will create a ticket and need verify the data which is on the ticket.

2. There are some configurations ( like rules) which will used to create behaviors of the transactions and have to verify the transactions will work according to the rule.

3. Whether the ticket is created for the correct user.

4. There are notification rules as SMS, Email and Letters, and needed to verify whether the correct notification rule is triggered. 

5. Verify if the ticket is automatically closed if outstanding or arrears is fully settled.

For automate this kind of scenarios what is best testing methodology ( tools, BDD vs TDD, framework type) should I use for create the framework and writing the test scripts.

Hope you got the above.

Could you please share your thoughts on this. It will be grateful. 

Thank you.

The Common Large Question

Listen to any Test Engineer who wants to start the automation, more or the same this will be the voice and words.  I was here and at times I will land up here when I have to pick the tooling part.  How I drill through this landscape and accomplish the mission will define who am I.  

This is the strategic part of testing and automation.  If this is firm to support the context, it can bring the values.  Else, I keep adding the costs to myself, the project, and the business.  The execution of the strategy will happen with what we pick; but, did it serve what we expect?

I want to demonstrate and share strategic thinking and apply it to arrive at an approaching and execution plan.  Doing this best to the context, will leave me with minimal costs to handle and survive with.  At least, I believe it so.  Just to make sure you are with me till here, do you hear such a similar question from Test Engineers who want to automate or start the automation practice?  

To me, this is one such common large question that I come across when I listen to testers and me.  I say it large for this reason -- the number of times I hear this question is beyond the count I can keep in my practice.

Interpretation of the Need and Requirement

As I read the requirement written and the follow-up discussion on same in the TTT's Discord space, I learn:

• The present execution state of automation
• Looks like no automation is written for this web application
• What's the product?
• It is a finance application
• The web UI is the user interface for this application
• Is there any other user interface for this application?
• Not sure
• What does the product do?
• As said in the requirement,
• The majority of business use case and function flow is to do with scheduling service
• This scheduling happens automatically that is the system takes care of it using the input from the user -- which can be seen as one transaction
• There are rules which can be called configurations
• These rules will define how and what the transaction must do
• Need to validate if the transaction adheres to the set rule
• Need to validate if the ticket is created to the right user using this rule
• Need to validate if the notification rule executed is right
• The different types of notifications available
• SMS
• Email
• Letters
• Postal letter?
• Each transaction will create a new ticket
• Validate if the ticket is closed by the system on outstanding credit or arrears is completely settled
• Technology and Tech Stack
• Java and MySQL is used in this web application
• Could be, 
• The backend is written in Java
• The database uses MySQL
• But the system as this that involves scheduling and notification will have queuing, caching, messaging, and searching operations
• The tech stack used for this is not mentioned in the requirement
• Imagine a system that has to initiate the postal letter notification
• This is something which involves not just software but also the people interaction very much
• Help Expected from Community
• As written in  the question posted
• "For automate this kind of scenarios what is best testing methodology ( tools, BDD vs TDD, framework type) should I use for create the framework and writing the test scripts."

Microservices and Where to Automate

I guess this product should have been built using Microservices.  When I say Microservices, I mean, a small, modular, replaceable, independently developed, and independently deployed software application that is responsible for performing one function within a large system.

An illustrative representation of Microservices system

When I have to pick the automation in the large system built using Microservices, I will know what is that I want to accomplish from automation?  And, this is one of the primary questions to be answered irrespective of what the system is and how it is built.

Further, this leads me to know and understand:

1. What data do I need to test and validate?
2. Where to validate data?
3. How to validate data?
4. Which is faster and more efficient when validating the data?
5. What is the expectation when validating the data?

This does not rule out or hold up any layer of the system.  In fact, this helps to know the importance, priority, and criticality of the layer (seam) where I have to start or have my tests concentrated.

I have missed showing the "data" within the hosting area in the above picture.  With that, I ask where should I test and automate to look for what I want to learn.  When I have this picture of my system in mind and how the microservices interact with the tech instances, it directs me to talk with appropriate team members and get started.

Getting Started to Automate in this System

Reading the problem statement and what is expected, it is nowhere mentioned by the tester -- how to automate on the web UI or API or other tech instances of the product.  But, the tester has mentioned the flows and their outcome which should be covered in the automation.

If I had to start testing and automation, I will do the below and assess:

• APIs:
1. I look at how good and deterministic is the APIs in processing the requests
2. Can I take the response of API to infer the system looks to function as expected
3. If yes, just this has to be taken by the client interface and display the result?
4. Can I test the notifications at this layer with help of automation?
5. Can I test the rules at this layer with help of automation?
6. Can I test for the configuration here?
7. Can I test for the ticket creation and its state updates?

• Web UI:
1. Do the contract between the web UI and API is obliged or does it have any problem?
2. Is there any JS feature on web UI that takes and processes data further?
• If yes, do I need to validate this to understand the correctness degree of the data to decide?
3. Or, the web UI just accepts the response and display the data?

• Databases, Queuing, Messaging, and Caching:
1. Is there a need that I should be testing and automating at this layer to decide on the data displayed?
2. Should I test here in association with other tech instances or in isolation to know the data?
3. Is testing of Microservice and its endpoint is sufficient?  Or, should I also include these tech instances in testing together with Microservice?

This will set me on the path now to decide where to start automation so that it assists and speed up my testing.  If the Web UI consumes the data from the endpoint and display, I will begin automation at the API layer.  This is simpler, faster, and can help me to figure out the problems distinctly at web UI and API layer.  

If I started automation at the web UI layer to determine the correctness degree of data, maybe I will have to spend more time to automate in this context.  If I have the freedom to go ahead, I will update the team and stakeholders on why I'm going ahead with this decision.  Or, if I have to present it to stakeholders and then start, I will present the work with data i.e. time, what is covered, and how much along with what is not covered and why.

That said, I'm not saying -- I do not automate at the web UI layer.  I'm saying, where will I start and why.  Making sure one part of the system is deterministic will help to figure out the problems in the other parts of the system.  The question from a fellow test engineer says, a requirement of automation for web application and not automating the application on web layer.

The tools and methodology come with culture and practice followed in the project.  At least the test engineer will have choices on the tools and library side though not at methodology.

The Never Ending Execution and Code Smell

 

I write this post to share my learning with all Test Engineers and not just be confined to Testers in a group.  I read the below question in the Telegram group - Testing Mini Bytes.  

A fellow Test Engineer in the group asked this question by sharing the method where he experienced a problem.

Question:

This is going on infinite loop can someone let me know why the loop the condition is not working

Pic: The method whose execution is not ending

Here is how I approached analyzing the stated problem and asisted the fellow engineer:

1. Read and understand the problem description
2. Understand whose problem it is and why from the description if the detail is available
3. Know the Programming Language used
4. Read the code and understand its flow
5. Looked for the debug statements
6. What is declared and how, why, and what does it hold?
7. Check if any state exists and what it is
8. Logic check for condition and flow
9. Check on maxAttempt count
10. Syntax check for condition and flow

Now, I look at how the condition check is being done:

1. The operator used in condition with the operands
2. Declaration
3. Assignment
4. Updating
5. Referring

In this case, the condition has IF and ELSE blocks.  The IF block has a check on the status and maxAttempt count.  The IF condition has logical AND between these two.  And, the ELSE block has a check on maxAttempt count alone.

What I infer is:

1. The operator != should not be a problem in this case
2. Though JS also has !== operator, != should work in this context; !== also works if used
3. Could be the status is not yet COMPETED or COMPLETED
4. But, the maxAttempt will not be less than 5 in the first check
5. maxAttempt is incremented
6. And, the same having this IF and ELSE block is called
1. Doing so, the maxAttempt will be reset to zero
2. Note that maxAttempt was incremented before resetting it to zero
3. In this method, maxAttempt will always be zero that is less than 5
7. Irrespective of whether the status is COMPLETED or not, this execution will run into a loop

Not having the debug prints or console log for status and maxAttempt, it won't be obvious why this execution is going on & not stopping.  It is a kind of recursion experienced that is not intended to be here.  Unless short of system resources, this recursion continues.

Though I see the condition check logic can be written in other approaches, I did not get into how it can be done.

Learnings

• Use the debug statements or console logs; it helps in debugging and understanding what's happening
• Code Smells
• Multiple Point of Failure
• The place where the maxAttempt is declared and initialized
• How the method is being called within it
• Long Method
• Though the method is simple in what it does, it involves multiple operations
• Also, the method calls itself within it
• It complicates by calling itself; breaking this method is good
• One method one task

The method looks simple but how it flows within by resetting maxAttempt to zero.  And, a recursion kind of nature makes it uneasy to follow when there are no debug statements.