What Test Engineers Forgot By Saying "Out Of The Box"?

When I started my testing career, I read the lines which said, think Out of the Box.  My then manager said the team to think and work, out of the box.  

I see testers practicing, working, testing and automating by saying, out of the box.

Wait! When said boxes, what comes to one's mind is Black Box, White Box and Grey Box.  But, I tell you, there exists no such boxes.  These boxes are imaginary; it was termed probably to give an analogy, to see and know the ways to test the product's code and infrastructure.

I do not tie myself to these imaginary boxes, but, it is a useful analogy.  After all, when said Out of the Box, which box are we referring to?

And, what is that I'm missing by saying and doing out of the box?

What Have we Forgotten?

While me talking and saying *out of the box*, I'm not letting myself to think Inside The Box, for first.

If I cannot know what is *inside the box*, how will I know what is outside the box?

If I cannot see, think and explore for what is *inside the box*, how will I know what is outside the box?

I see, we have forgotten "Inside The Box" by saying and hearing "Out of The Box" for decades.

What's inside the box which you are testing?  Does it have just functionality, though the functionality is the heart beat of a system?  Okay, have I explored the functionality in all possible dimensions?

Almost everything in the box is associated with functionality.

When I'm not said to test for what is inside the box, I will be lost with Out of The Box.  Do you?

If I explore Inside The Box, fore sure, it will help me to go beyond functionality.  What all I can do inside the box?  What all I can evaluate inside the box and how?  That is, I think of performance, security, usability, observability, traceability, monitoring, and you label next.  All these are associated directly with the functionality.

All the boxes in software testing are imaginary.
We have forgot to see and explore Inside the Box.

Inside the Box, helps you to identify and observe,

• Why these exists?
• Why these are interconnected?
• How these are interconnected?
• How are they communicating?
• What are they communicating?
• To whom they are communicating?
• When are they communicating?
• When they fail to communicate?
• How fast and reliably they can communicate?
• The programming, code, protocols, tech stacks, libraries, money, ports, cables, people, etc.
• What they are holding and where, how and why, when?
• And, more ...

As you go Inside the Box, this space grows in multi dimension.  You will realize what is out of the box and what is inside the box.  You will consistently tune yourself to fit in any imaginary boxes with your tests.  From there on, you will explore, test, automate and talk to all the imaginary boxes and system[s] you are interacting with.

If I cannot see what is within, how can I see what is outside?

The Reality, Pain, and Cost

This is my observation.

• When I ignore Inside the Box, I will continue to talk with more focus on vocabulary, terminologies, schools, process, advocating on one aspect of the discipline and think this is craft and craftsmanship.  
• I will be stuck here and pass on the same to upcoming generation of software test engineers and practice.  
• This is the reality and painful part. 

Is talking and working on vocabulary, terminologies, schools, craft, craftsmanship and advocating that one aspect is testing, wrong?  

• No, it is not wrong.  
• But getting lost here indefinitely is beyond wrong.  
• The cost of this is huge to the craft as a practice and to the software test engineering discipline.
• As a result, the upcoming test engineers and generation will pick whose voice, approach and content is more opted.
• If observed, this is not the case with programming and programmers.
• I see, the programmers challenge most times and everyone who pushes or sells the thoughts. 
• Why? I see, they are trying to learn and understand from inside the box for first.

Unfortunately the smart people who are with role and title of influencers, thought leaders, keynote speakers, seniors, directors, VPs, ambassadors, and advocators of a thought and schools are lost here.  My question to them and me is, why not we explore Inside the Box?  Why just confined for propagating the philosophies and one's ideas alone ignoring what is Inside the Box?  The philosophy is a need; I do not say we should not ignore it.  Philosophy alone do not pay one's bills and fill the stomachs.  I wish, I had learned this lesson in the start of my career.  The business is a philosophy by itself and it will make use of or ignore any other philosophies as and when needed.  I hope and wish, the test engineers will interpret this.

I see this is one of the key reasons why the software testing as a discipline and practice we did not gain the recognition and importance as programming the product and solution.  Though we work closely with the product code and the test code, we miss our craft and names being called out.  This is reality and painful.

To end here, if I'm not seeing and thinking for what is Inside the Box, how will I excel in Out of the Box?

I Have a Programming Book in My Software Testing Books List

A Question From Mentee

One of my mentees asked for the books to understand Software and Software Testing better.  She shared the list of books to which she is referring.  She asked,

What other books and resources do you recommend to me for learning and understanding Software Testing?

I want to share my thoughts by talking a bit about boxes of software testing, books, and personas.

Software Testing, Books, and Personas

Read this blog post (Black Box in Every Other Box of Software Testing). It talks about the different boxes in Software Testing.  On reading, do you see a Black Box in every other box?

I classify the books and resources on Software Testing per the below classifications.

Pic: Classification of Books & Resources for Software Testing

I keep the Programming book in Assistive classification.  I have it in my first three books.  How can I test a software system if I do not understand how it is programmed?  Maybe, I can do testing to some extent. But,

• What will I miss if I do not understand how it is programmed?
• What will I miss if I cannot make sense of the code?
• What will I miss if I do not know how to use automation to help my testing?

I consider the different personas to aid my testing.  But, I do not see the programmer persona of writing the code for the system I'm testing.  How to use the Programmer persona to test the system?

What stops me from considering programming as one of the characteristics of a persona?  What can influence and help me to think and include this persona?

I'm not knowing [or ignoring] the first circle of persona (programmer).  But trying to think of different personas to include in my testing strategy.  Is this rational, logical, sensible, and appropriate?

If I practice programming to the extent of reading and understanding the code, I can think of risks to a larger extent.  This is helpful.

If I practice programming skills to the extent where I write automation, I can think of how to automate, how much to automate, and where to automate in that context.  It is helpful.  Isn't it?

If I practice programming skills to the extent where I can discuss confidently with the programmer about the possible risks, it is helpful.  It will also help me to debug and test investigate better.  Isn't it?

This will change how I look at the system and how I test the system with my testing and automation.

Book, Programming, and Language

• In your list of Software Testing books and resources, have a programming book
• Testing software without knowing or understanding programming will always limit us sooner in our Test Coverage, Test Strategy, and Automation Strategy
• It will always limit our understanding of the system we are testing
• It will always limit us in knowing the potential risks and problems inside and outside of the system
• It will limit our perspectives
• For example, 
• When and how the functionality of a feature can be vulnerable?
• When and how to decide how much sampling is enough in a context?
• When you are building a mindset and thought process in programming, pick a programming language to practice the programming
• Start small
• Start simple
• But, start, and then continue

It is always good to derive relative learning and analogy from different practices and study areas.  It helps to register one's learning.  But it should not happen at the cost of avoiding and ignoring the practice of programming by a Software Test Engineer.

Programming is very closer to the software system which we build and test.

Do not get into the thought of whether it is compulsory to be aware of programming for testing software.  This is procrastination probably due to fear.  Face it!  Fight it!

I suggest "Head First Programming" for programming for one who is starting to practice programming.  This book is self-descriptive and communicative.

Start practicing the programming at your own capability; but, do not give up; continue the practice.

Upskilling yourself here will put you on the competence line.

I have a Programming book in my list of Software Testing books and resources.  Do you have it?

Black Box in Every Other Box of Software Testing

 

Modeling Software Testing With Boxes

The fact is something that is not put to scrutiny or questioned much and often.

As a fact, the Software Testing is explained to us using boxes.  That is,

• Black Box
• White Box
• Grey Box

Is this wrong? No, it is not wrong.

There was a need to explain for one how to visualize -- how a person would interact and interpret the software system when testing.  The analogy of these boxes helped and helps.  These boxes are mindsets.  In a way, these boxes are like models to interpret the different ways and approaches we take in Software Testing.

I see, we are seeing Black Box in every other box.  Maybe, this is limiting one not to think to learn software testing in other than a black box mindset.

If you ask, are we not automating? Is not that a Grey Box?  Very much, we are a black box mindset when writing automation as well.  I include myself here.  I'm exploring how to break out of this and see the Software Testing.

Do programmers think of their code as different boxes?

• A programmer reads, writes, deletes, and views the data. events and state
• The programmer as well cannot see what's happening between the binaries on the electric circuits
• The programmer evaluates her/his code's testing via logs, debugger, and assertion for the data, state, and events
• Is this a white layer or box?
• Is it called white because one can see through the logs, debugger, and assertion?
• But that is still not a sight of binaries on the electric circuit, right?
• If one could see through the binaries, we should not be having race conditions, out-of-memory, and unhandled exceptions
• Isn't so?

This makes me feel, there is a Black Box in every box and we are largely confined to this Black Box.  

Exploring to step outside this box helps to understand the testing, software testing, system [and software] under test, and what is needed to test better.

Inside Story: What is in my talk for you at API Summit 2021?

I'm excited to say I'm presenting my talk at API Testing Summit 2021, this 8th May 2021.  Registration is free and you can register here.  

Get your testing team and fellow engineers to the summit. I see the unique value it brings with an opportunity to network with Test Engineers. Before reading further, register here and mark your calendar.

Does Black Box Exist?

The term "Black Box" is very much known to Software Test Engineers. Almost everything is a Black Box in relativity? How can we look into the black box or box of any color? What is the inside story of a black box?

Is API a Black Box?

To the consumer of an API, in a way, it is a black box. The consumer usually does not bother what happens within an API. All the focus is on processing the given information and responding appropriately.

As a Tester, do I have the below questions in me?

1. Should the API consumer know what happens behind and beyond the exposed endpoint?
2. Should the API consumer know what happens behind the exposed endpoint? 
3. Should the Test Engineer see an API as a black box?
4. Should the Test Engineer see what this black box constitutes and why?

Inside Story: Scratching the Black Box – API

I'm trying an experiment of analogy and demo. It is a 30 minutes talk and how can I make it? I'm working on it. I'm keeping it as short as possible as I have to demo and draw an analogy with the current pandemic day. 

In the 30 minutes' brief talk and a short demo

• I try to visualize how we can scratch the black box - API
• Dive layers down in learning and testing the API in a context. 

Who are the Audiences of this Talk?

This talk focuses on the fundamentals. Going advanced means getting deep into the fundamentals. To advance high and further, consistent learning of fundamentals is a must.

I classify my audiences of this talk in two categories as below. Do not forget to get your fellow Testers to this conference.

1. In Testing:

• One who is wanting to start the practice of API Testing
• One who is practice API Testing and wants to debug better

2. In Current Day:

• One who is looking at the happening pandemic
• The learning of API that we know but not monitoring enough

About the Conference

API Testing Summit 2021 is brought to you by

• Agile Testing Alliance
• DevOps++ Alliance
• CP-WST
• QualityKiosk

Conference web page

• https://apisummit21.agiletestingalliance.org/

HashTag:

• #APISummit2021

Me in Speaker Interview Series

• https://youtu.be/TfcYNQVU7C0

For other speakers interview, look at the right side of this below page:

• https://apisummit21.agiletestingalliance.org/ravisuriya-eshwara/

Why the Wait?

I'm waiting to listen to the other speakers and learn. Why you wait? Register and get your fellow testers and team.

See you at the conference and my talk -- Inside Story: Scratching the Black Box - API.

After the talk, you decide what box and layer of an API to uncover in your testing.

Challenges in Building the Skilled Software Testing Team and Testing Leadership

In testers meet, I was asked, "I happen to see overlooking the value of building the testing team and contributions team make to shipment. Do you see same in your work? How to solve it?"

Before I went ahead to discuss on this, I had silent in me.  I see certain problems are in the culture and mindset for first.  Bringing a change here is not that easy - culture and mindset. It also applies to testing teams. It also holds good to any others (or teams) interacting with the testing team.

Taking the analogy of skilled carpentry work (say, "pepperfry") - we will see how this exists not just in testing but anywhere. But in testing, it is not changing after decades too. That's the problem causing unbearable costs and not the actual problem. It's not being attempted to understand by who needs the testing, in my opinion for today.

A skilled and experienced carpenter can do her/his work quickly and with values, right? How many carpenters exist today who can do job at least to level of saying this is the minimal must value and job, I need without the supervision and initial assistance from skilled carpenters.  If seen, the assistance in carpentry work is not for weeks or months.  It takes longer time depending on the skills, mindset, passion and attitude of person learning the carpentry.  Now comes the context - there is a list of orders coming in; the skilled carpenter has people who started carpentry and the people who can do job on consistent supervision for 'x' period of time. If the skilled carpenter puts her/his time in doing the work day in-out to deliver the orders alone, may be orders delivery will be met but with diminished values in work eventually? How long the skilled carpenter can do this i.e. all alone? The skilled carpenter should not assist her/his fellow carpenters and help herself/himself or people who will be hiring them for job?  I learn, the skilled carpenter will assist and give her/his more time in closely watching her/his fellow carpenters while chipping out the core part of the orders.  What if the fellow carpenters quit and join elsewhere? That cannot be stopped by any means. Due to people quitting does one has to lower the standards of one's work? Do the furniture brands say we lower our standards of product as we find carpenters quitting the job? NO!

Relate the same to Software Testing and Automation.  Including me, we have people in software testing -- who knows jargons; who knows how to use what they know in shallow or not know at all; and starters. Here we will have serious people who are interested and also not interested.  Considering the current time (future as well hopefully) of people or team who needs the testing team, isn't that a skilled tester job to build a team on which they can sustain and rely in coming days?

This long activity i.e. building of testing team can take years. And in my experience when had a people with right attitude, consistent efforts and passion - it can take close to 2 to 2.5 years for one.  In these 2 to 2.5 years, we can see people tackling the testing problems if practiced well. But people do change job in 2 years or so, now should I invest in it?  This is one of the invest-cost-value problems which has made Software Testing to take back seat.

It has gone to least priority when considering of having the skilled testers and testing team. Because, practicing testing is not done by most. Later saying this is testing to people boarding the team/org do this and getting the lower returns and values for team & org. Eventually blame the testing and testers. One simple example, we are aware of Design Patterns in programming and know several books/authors who has authored.  Do we know the Test Designs and design techniques apart from black box, white box, grey box, and any books or authors who have authored on same? Blinking of brain starts here!  I feel there is no need of better example than this.

Everyone has experience but that does not mean they are practitioners.  This is the trouble with people boarding into practice of testing. They happen to hear and work upon hearing - just do this, like this, this much, and as this. Write your test cases, automate all, bug reports and everything here. If that can be said in ease, it can be done in much more ease. Why hiring of the testers and having testing team?  Now you know the trouble and pain points of building the testing team and testers?

The expectation from the skilled (carpenter) tester is X+1, but the people who gave the job/orders see it is y-X and not far close to X.  This is the exact case with people or team hiring the testers and wanting to see the "proportional" value right away in the product and to the organization.  It can be achieved, if considered just the skilled (carpenter) tester and for a shorter period of time. But is that the expectation of people who pay for the all carpenters (testers) doing the job in a business?

To see the value of a test, it can happen in a release or few releases. To see the value of testing being done or not done, it can take few releases.  To see the value of having the skilled testing and their work, on which people rely, it will take years.  I see same applies to programming team as well. But in programming space we have people who practices it and then assist boarding people.

In the testing space, we don't have practicing people to assist the boarding people and existing people, in the right way.  This contributes to laggard and continue giving a low mark to the testing team and testers.

Now is it -- problem of the industry; problem of the org; problem of the people or teams wanting to have testing teams; problem of the testers itself?  Whatever, the impacted entities are -- industry, people or team who want to have skilled team and of course testers too.  Help the testers, testing teams and testing practices you have in your org. By this, you are investing. When investing know if it is done in a right way and on right stuffs. Otherwise, the investment made today in testing can become blocker cost.

If a skilled (carpenter) tester is working on building the future carpenters by giving her/his assistance, it is not a simple job! That's the bigger value. Doesn’t the business find returns one day out of this assistance? I say it is not simple job.

The skilled (tester) carpenter knows that he/she is not doing in full what she/he has to do in a given context while continuing to assist other carpenter.  But negotiate in open mind with stakeholders by bringing the understandable and mutually agreeing communication - what is expected immediately out of carpentry work given or the orders that came in.  If this is done, half-the-problem is solved when you have skilled testers.  Often, this will not happen and skilled tester too fail here. 

The communication of expectation for a carpentry work coming from different people; to whom should the carpenter look upon and deliver the asked delivering the work?  We testers and testing team have this problem as well in getting our work set and delivered.

How to solve?

1.   It needs a leadership in testing at org and in teams.  The test leadership, if they are testing practitioners it is good and boon. But hands-on practitioners might find time constraint to be in management and hands-on delivery. Yet this is do-able in my experience.

2.      Testers need to pull up themselves and bring the change in-out.

3.     Culture and mindset problem, needs better way of solving and not the software testing way i.e. how software testing solves problems.  Unless having the faith in software testing and how it solves the problem, it cannot be used as a relativity in learning to learn and solve the problems.

4.   At the bottom, skilled practitioners will be on the hit list most times. Unavoidable! But that should not put down the Software Testing community and practitioners growing. One day you will be recognized, remembered and highly valued for what you have done.  Just make sure you don’t incur big costs personally by doing this activity. Balancing is the key here and it comes only by incurring the costs while doing this activity

If not bothered about

·        need to transform into skilled testing practitioners;

·        org not having/wanting the efficient testing teams;

·    continuing the shallow and low returns testing & automation, which is highly accepted, paid better for doing that and you don’t care a penny anymore about it,

then do not attempt solving this problem for you, your team, and your org. It will never be solved. It will just increase the troubles to all.  One day it will be realized and it will be picked to solve by stakeholders upon having the costs.

Note: I admire how the "pepperfry" crafts its furniture. I wonder why the same cannot be achieved by carpenters who do carpentry while constructing the house. I'm left with many thoughts on this and I try in discussing them with a tester in me.