I Have a Programming Book in My Software Testing Books List

A Question From Mentee

One of my mentees asked for the books to understand Software and Software Testing better.  She shared the list of books to which she is referring.  She asked,

What other books and resources do you recommend to me for learning and understanding Software Testing?

I want to share my thoughts by talking a bit about boxes of software testing, books, and personas.

Software Testing, Books, and Personas

Read this blog post (Black Box in Every Other Box of Software Testing). It talks about the different boxes in Software Testing.  On reading, do you see a Black Box in every other box?

I classify the books and resources on Software Testing per the below classifications.

Pic: Classification of Books & Resources for Software Testing

I keep the Programming book in Assistive classification.  I have it in my first three books.  How can I test a software system if I do not understand how it is programmed?  Maybe, I can do testing to some extent. But,

• What will I miss if I do not understand how it is programmed?
• What will I miss if I cannot make sense of the code?
• What will I miss if I do not know how to use automation to help my testing?

I consider the different personas to aid my testing.  But, I do not see the programmer persona of writing the code for the system I'm testing.  How to use the Programmer persona to test the system?

What stops me from considering programming as one of the characteristics of a persona?  What can influence and help me to think and include this persona?

I'm not knowing [or ignoring] the first circle of persona (programmer).  But trying to think of different personas to include in my testing strategy.  Is this rational, logical, sensible, and appropriate?

If I practice programming to the extent of reading and understanding the code, I can think of risks to a larger extent.  This is helpful.

If I practice programming skills to the extent where I write automation, I can think of how to automate, how much to automate, and where to automate in that context.  It is helpful.  Isn't it?

If I practice programming skills to the extent where I can discuss confidently with the programmer about the possible risks, it is helpful.  It will also help me to debug and test investigate better.  Isn't it?

This will change how I look at the system and how I test the system with my testing and automation.

Book, Programming, and Language

• In your list of Software Testing books and resources, have a programming book
• Testing software without knowing or understanding programming will always limit us sooner in our Test Coverage, Test Strategy, and Automation Strategy
• It will always limit our understanding of the system we are testing
• It will always limit us in knowing the potential risks and problems inside and outside of the system
• It will limit our perspectives
• For example, 
• When and how the functionality of a feature can be vulnerable?
• When and how to decide how much sampling is enough in a context?
• When you are building a mindset and thought process in programming, pick a programming language to practice the programming
• Start small
• Start simple
• But, start, and then continue

It is always good to derive relative learning and analogy from different practices and study areas.  It helps to register one's learning.  But it should not happen at the cost of avoiding and ignoring the practice of programming by a Software Test Engineer.

Programming is very closer to the software system which we build and test.

Do not get into the thought of whether it is compulsory to be aware of programming for testing software.  This is procrastination probably due to fear.  Face it!  Fight it!

I suggest "Head First Programming" for programming for one who is starting to practice programming.  This book is self-descriptive and communicative.

Start practicing the programming at your own capability; but, do not give up; continue the practice.

Upskilling yourself here will put you on the competence line.

I have a Programming book in my list of Software Testing books and resources.  Do you have it?

Black Box in Every Other Box of Software Testing

 

Modeling Software Testing With Boxes

The fact is something that is not put to scrutiny or questioned much and often.

As a fact, the Software Testing is explained to us using boxes.  That is,

• Black Box
• White Box
• Grey Box

Is this wrong? No, it is not wrong.

There was a need to explain for one how to visualize -- how a person would interact and interpret the software system when testing.  The analogy of these boxes helped and helps.  These boxes are mindsets.  In a way, these boxes are like models to interpret the different ways and approaches we take in Software Testing.

I see, we are seeing Black Box in every other box.  Maybe, this is limiting one not to think to learn software testing in other than a black box mindset.

If you ask, are we not automating? Is not that a Grey Box?  Very much, we are a black box mindset when writing automation as well.  I include myself here.  I'm exploring how to break out of this and see the Software Testing.

Do programmers think of their code as different boxes?

• A programmer reads, writes, deletes, and views the data. events and state
• The programmer as well cannot see what's happening between the binaries on the electric circuits
• The programmer evaluates her/his code's testing via logs, debugger, and assertion for the data, state, and events
• Is this a white layer or box?
• Is it called white because one can see through the logs, debugger, and assertion?
• But that is still not a sight of binaries on the electric circuit, right?
• If one could see through the binaries, we should not be having race conditions, out-of-memory, and unhandled exceptions
• Isn't so?

This makes me feel, there is a Black Box in every box and we are largely confined to this Black Box.  

Exploring to step outside this box helps to understand the testing, software testing, system [and software] under test, and what is needed to test better.

JSON for Software Test Engineers - 101

 

In the practice of Software Testing & Engineering, I became aware of the below:

• UI (User Interface) is not just GUI (Graphical User Interface)
• UI is an interface through which a user interacts with a system
• UI is one of the touchpoints
• Data we exchange between the two systems is also a UI
• This data can take different formats
• Text
• HTML
• Image
• Video
• Audio
• XML
• JSON
• YAML
• PDF
• custom-defined data format
• and, more

Note: Refer to the MIME types and know what type of media is supported and used in the system you are testing and automating.

Testing Through Data

When the data can be used as a UI, it also means one can do the testing and automation through data using data.  From this perspective, it is important to know the data format.

I see JSON is one of the common data formats we use to store and exchange data between two systems.  Understanding how the data is structured in JSON is important for a software test engineer to think and evaluate the testing and automation through the data.

JSON For Test Engineers - 101

I documented my understanding of JSON and how to interpret it in the gist here. This markdown file has my learning.

Test Automation Pyramid and Its Multiple Misinterpretations

 

I Decided to Write this Post

I thought over a few months, should I write this post. I said, yes to myself for writing this post.  Before, I proceed, I want to express this:

Hello Mike Cohn, thanks for giving Test Automation Pyramid visualization to the software engineering community.  I see your thought process, experience, and work, in this model. This model's assistance to understand software systems is evident today.  

Again, thanks for giving this model as a metaphor for us.

What I See?

I see a risk for the Software Testing community especially when we use the word The Test Pyramid without knowing what it is and does it make sense. 

To keep you till the end of this post so that you get what is my point, I tell you this:

One of the common entities among Testing and Automation is the tests.  How the test's intent is expressed and executed in the testing and automation, is not similar; they are different and have to be different.  If similar, then it leads to misunderstanding -- automation as the only way of testing; or, the execution of tests by a human as the only way of testing.

Why did I Say "Yes" to Myself?

• The metaphor which Mike Cohn came up with is helpful
• I see, Mike Cohn knew what he is trying to say and it makes very much sense in today's Software Engineering
• His book Succeeding With Agile got published in 2010
• It has a chapter with title Quality
• I insist, that anyone who is involved in software development should read this chapter if cannot make the time to read this book
• This chapter talks about the pyramid
• The Test Automation Pyramid
• Mike Cohn came up with this concept
• This acts as a helpful metaphor and a heuristic
• In 2010, I don't know how much did it help then; but this model is a beautiful and simplistic explanation of how to categorize the software system's layers for test code
• In 2023, it helps everyone who is involved in the development of software system
• I have witnessed how software engineering has grown in the last 23 years
• Reading what Mike Cohn has to say about it, helps to clear the misunderstanding of The Test Automation Pyramid to The Test Pyramid, for first

Today the word "The Test Pyramid" is very common among people who are into software development. For this reason, is it misread and communicated?  Maybe!  The software engineering community refers to this model as The Test Pyramid and not as The Test Automation Pyramid.

I want to share my thoughts on the risks of calling it The Test Pyramid.

• I strongly see the software engineering community and importantly the software testing community need to know it

So, I have decided to write this post.

Quality Is a Team Effort

Mike Cohn, said this in 2010 in his book Succeeding With Agile. 

Today, who does not claim "We are Agile"?  Are your team and organization, Agile?

Yet, we see discussions on who should be owning and responsible for the quality while we are in 2023.  Especially the software test engineers ask this as they are said responsible for the quality of the software systems they are testing.

The chapter Quality has a section "Quality Is a Team Effort"; this section has the below lines:

Acquiring new testing skills, learning how to apply them within the strict timeboxes of Scrum, and paying off technical debts are the responsibility of the whole team.  These are not challenges to be sloughed off onto the testers.

Quality is everyone's effort and responsibility.

Test Automation Pyramid

Automation is one of the ways we do the testing.

Here are some of the tests (actions) that can be done through automation and asserted:

• Which are repeatable and can be asserted
• Which are run to setup, tear down, configure, create, edit, read, and delete the resource 
• Which are monitoring to inform the change in state/event/data
• Whose execution is in a defined order and trying to assert the expected outcome

Some tests are better evaluated via automation and with the help of automation.  Some tests are better evaluated by the execution of a human.

• Both are important and necessary in today's Software Engineering and Software Development

Back in 2010, the boxes were monolithic; the microservices were a theory.  Today, we are thinking beyond microservices, containers, serverless and more.

Given the complexity of  Software Systems, today, we need a model of how we can isolate it into categories and layers.  So that it is simple to learn, understand, to categorize the development of code & test, and execute the testing of same.

This is where I see the usefulness of a model given by Mike Cohn.  It shows the layered architecture pattern in a way, for a [part of] software system:

• That we are about to understand, and 
• Figure out the strategy for testing using automation in respective layers

Now, do you see the importance and help of the Test Automation Pyramid model?

Mike Cohn expressed the three layers in Test Automation Pyramid. It is as below.  I have copy pasted this below image from his book Succeeding With Agile. That way, I keep the original thought and picture of the model as given by the author.

Pic: The Test Automation Pyramid from Mike Cohn

Pic credits: Mike Cohn

Benefits of Test Automation Pyramid

• It helps to relook into the automation strategy
• It says, we need a better strategy to automate by showing us to categorize the tests in a layered pattern
• It helps to learn how the two different layers can communicate and can be used together in automation 
• For example, the service and UI
• So that tests are isolated and maintenance is manageable, while the deterministic feedback obtained from the tests automated is value-adding.

I have heard Dorothy Graham say she did automation when she started testing.  I see she has 40+ years of experience in Software Engineering & Testing.  So, automation is not new.  It is evolving.

This Test Automation Pyramid describes the automation which we talk about in day-to-day engineering to be more understandable and communicative.

Is This a Test Pyramid?

I understand, Test Automation Pyramid is not The Test Pyramid.

It is a pyramid for the tests carried out through automation.  

Test Automation Pyramid helps to identify, design, categorize, relate, map, and use the outcome of tests from another layer.  This benefit can be sought in both, that is, testing through automation and testing by a human.

This is a risk I see

• The Software Engineering and Software Testing communities are learning that automation is the only testing and the only way to test.
• No, it is not the only way to test and it is not what is testing all about
• Automation is one of the ways we do our testing and has its limitations
• Automation is a subset and a part of Software Testing
• Automation exists to assist Software Testing and to do better testing as a whole; not to replace the testing; not to claim it is the only testing and way to test in every dimension and context
• This is a problem for the upcoming generation of software engineers, and those engineers who take Software Testing as their full-time practice and job
• This sends a message that automation is testing and testing is automation
• This will be the problem to the practices of automation and testing in Software Testing
• For example, when I talk to test engineers and say, Test Automation Pyramid, I hear they are not aware of it; all they are aware of is The Test Pyramid of three layers used for automation
• As a result, there will be shallow results and benefits from both automation and testing with costs and short of value
• Here, automation and testing cannot complement each other effectively when used together

Calling it a Test Pyramid and referring it to just to talk about automation, is not right technically and logically.

I see Mike Cohn knows the differences between the value of Testing and the value of automation in testing, as different activities from his work and experiences.  I believe, for this reason, he labeled this model explicitly as The Test Automation Pyramid.

I did not find any references for Mike Cohn renaming this model as The Test Pyramid.

Also, calling it a Test Pyramid and the top of a triangle having a scoop or cloud with the name "Manual Testing" is misleading.  I see this is one more cause of confusion.  Does this triangle say to automate at every layer and just involve a human at the GUI interface for testing?

• No! We, humans, test at the service layer as well and also automate here
• At the unit layer,
• a human manages to setup/configure at the seam or/and use test doubles, stubs, mocks, and what is needed to execute the tests via code
• If you see, one pyramid, misinterpreted and misrepresented in multiple ways.

I learn, as a community, we are misinterpreting the Test Automation Pyramid model [work] and calling it by another name.  Is this right?

If one sees no difference between testing and automation, fine!  Then everything is applicable and looks alike.  Then calling Test Pyramid and Test Automation Pyramid does not make any difference.

But, actually, it is not the same [and similar].

The common thing between automation and testing is the tests.  The purpose, nature, procedure, design, strategy, and intent of the tests differ in automation and testing.

It is Test Automation Pyramid

We Software Test Engineers, think [of] [the] tests as automation most times because we believe and are said the programming and code are everything in the software system.

• The code of a software system is one of the byproducts of developing a Software System
• So are the tests.
• The test code is one of the byproducts of automation and not the whole software system

A part of the test is automatable. Which part? 

• It depends on how I express my test with its intent
• So that I will know which part and aspect of a test can be automated

Automation in testing is one of the subsets of Testing, but NOT similar to Testing in every dynamics, dimension, and aspect.

Automation has its own subsets of dynamics, dimensions, and aspects of testing.  If we practice by learning Testing and Automation are the same and if we remain in this understanding,

• We cannot identify
• The differences between Testing and Automation
• The differences between the tests identified for Testing through Automation and Testing by human
• The values each offer along with its limitations.

Referring it to as a Test Pyramid and just talking about automation out of it, is not Test Pyramid.

I call and refer it to as Test Automation Pyramid and not The Test Pyramid.  Again, I thank Mike Cohn for giving this clarity in his model and its name.

Let us refer to Mike Cohn's model by the name and understanding to which he refers it with.

If you ask/say, "it is a model; can't we improvise it to one's context?" 

• Yes, we can and a model is not fit for all contexts
• Showing the same model picture along with the same name for its attributes and expressing a different thought is a problem
• It misleads one if she/he takes this model as a reference
• This will have chains and it continues

To end,

• Embrace programming
• Practice programming
• Embrace automation
• Practice automation
• Implement automation
• Call and refer to automation as automation
• Know what is testing in automation
• Know what is testing
• Call and refer to testing as testing
• Practice testing
• Know your testing in Software Test Engineering
• Call and refer to the model with its name given by its creator/author
• That name has a purpose and intent
• Know it; understand it; learn it

References

• The book "Succeeding With Agile: Software Development Using Scrum" by Mike Cohn
• https://www.mountaingoatsoftware.com/blog/the-forgotten-layer-of-the-test-automation-pyramid
• https://martinfowler.com/articles/practical-test-pyramid.html
• https://medium.com/tide-engineering-team/the-practical-test-pyramid-c4fcdbc8b497

The Unsaid Difference Between the Terms UAT and EUT

 

I read this in Telegram chat of The Test Chat:

I am listening to a very interesting conversation about testing. The bone of contention is - UAT (User Acceptance Testing) is same/or not to End User Testing. What do you think?

 

My Opinion as a Blog Post

I learn, writing this opinion of me as a blog post is of use and value adding to the community.  So I write this mini blog post.

The discussion around this topic do exist since I started my career.  Maybe, it will continue as well.  And, that is not a problem.

The concern I see is, how these terms are being interpreted in the discussion. And, this discussion will continue for years to come.

I attempt to share my thoughts on how I look at these terms -- UAT and EUT.

Introspecting the Terms UAT and EUT

Here is how would I start to understand these terms and learn.  I use the approach "Replace and Interpret".  When I experienced difficulty and ambiguity in practice of subject and concept, this approach crossed my mind.  Today, I use it consistently in my Software Test Engineering practice.

For example, I see the below few on replacing the word "testing".

• User Acceptance Automation
• End User Automation
• User Acceptance Driving
• End User Driving
• User Acceptance Singing
• End User Singing
• User Acceptance Programming
  
• End User Programming
• User Acceptance Vehicle
  
• End User Vehicle
• User Acceptance Utensils
• End User Utensils

I can keep identifying such replacements. I try to learn by replacing the words, idea, thoughts, or what is seen as fit to replace.  Doing so, I see the differences in what it means in different contexts.  Isn't so?

Prior to understanding and learning these words, I learn and understand the thoughts of people who are using these words and the differences they see.  The learning of this difference helps me much than the difference of the two terms -- UAT and EUT.

I see, the discussion on this topic and words has not changed much since 15 years.

Today, in the time of 10 days sprint (or two weeks sprint), do someone have an opportunity with time to carry these -- UAT or/and EUT?  Which product/system get the space for the UAT or/and EUT?

We have to learn these terms (UAT & EUT) from that system/product's perspective rather learning as generalization in Software Engineering and Software Test Engineering.

Do not start and establish the discussion and understanding of these terms by associating it to Software Testing right away; and, then call it out as a [generic] activity of Software Test Engineering. 

Introspect these words from the perspective and context of the product/system you are testing.  Look out for its necessity and purpose of being executed.  If you see these two are different activities for a system/product's context, it has a purpose for being different activities in that context.

This is my thought and opinion on these terms for now.

To stop here, I see the "Testing" is common to both [of these words].

Maven For Software Test Engineers - 101

 

Why 101s?

In the initial days of my career, I looked and referred to multiple sources to understand the technology stack.  As I progressed, I looked for someone who can help me in understanding what I'm reading and to clarify the questions I witnessed.  The understanding of concepts took time when I actually started using it and especially when debugging.

Today, if I look at that, I ask myself

• How can I make my learning experience a happy one?
• How can I capture the fundamentals and have a smooth revision of it when in need?
• What can I have for my reference when I'm implementing the fundamentals?
• How can I increase the speed in which I do all these?

If seen, I want to help myself to be better and effective when solving a problem by implementing what I'm learning.

Maybe, other software test engineers too have same question.  Don't you?

I'm learning new stuffs, and also unlearning what I'm learning.  I create the 101s for myself.  I see the software testing community can also make use of these 101s that I'm building.

Why You Need Maven 101?

• Today, it is a common sight that one quickly picks up a boilerplate code and run it
• Especially with the software test engineers, who wants to start/practice automation
• Pick a framework and run a command
• Execute the program successfully
• Or, get stuck in between and do not know why is that command
• Or, see an error with that command and not sure what that command is doing
• What is the difference between these commands and what does it do?
• As this, one can get into different contexts where one gets trapped; not blocked
• How to help self to navigate out from here?
• I see this 101s can come to help

Anytime, you picked a Maven project and built it?  For example, a Selenium code written as a Maven project.  Do we know what Maven is and why the Maven project here?  And, what these different Maven command means and what it does?  For having this understanding, this Maven 101 comes to your help.

Maven for Software Test Engineers 101

I have my Maven 101 here.  I refer to this source when I revise my fundamentals.  This can be of help to you as well and also can be a start point.

This has helped me to understand the different build life cycles and how a Maven command is all about the plugins.

I have provided the Reference section which is also a credit that I give them for helping me to unlearn and learn better.