Waterfall or Agile: Testing for Performance - Where to Start?

 

Do you understand the Agile?  I have shared my understanding here; give it a read.

The eighth question from season two of 100 Days of Skilled Testing is:

Can you share some best practices for conducting performance tests within an Agile development environment?

Best Practices and the Agile

The irony is, the Agile says, there is no best practice.  It asks, to tailor and fit the practice to the context so the continuous delivery and value is delivered consistently upholding the Agile's principles.  

Yet, we talk about the best practices in the Agile's context, like the eighth question asked here.

What is the effective way to test in the continuous delivery?

As a test engineer, how can I start thinking and testing for performance from the inception of a feature's thought?  I see, it is not hard to do so.  As you read further in this post, you will have a perspective and awareness to do it.

Performance in Waterfall and Agile

I learn, the performance is an experience.  It does not differ because of the Waterfall or Agile.  If the performance is not a pleasing experience, it will impact stakeholders no matter it is Waterfall or Agile.

But, the question when evaluating for the performance is -- where to start, when to start, how to start, and with what to start?

As of today, I do not see differences in the mindset and skills one has to have for testing of performance in Waterfall and Agile.  Could be the approach differs in certain phases here; otherwise, I see the same in both practices.

I will rephrase the eighth question to this:

What is your practice to evaluate the performance right from the start of product development in your project?

I do not want to wait until to hear -- the development is completed and deployed; now we can start running the performance tests.

What can I do as part of performance tests from the first day of development and first commit?  This is my intent and area to look in strategizing the testing and tests.

The Culture of Engineering

At the start and end of the day, when we developers start and finishes the work,

• How the work is done and why, is defined by the engineering culture practiced by that organization.
• The Performance Engineering of the software products and solution being built will be driven the by the culture practiced.

The Test Engineering and how we test and automate will be driven by the culture of engineering practiced in the organization.

Writing the code not just for building the functionality, but, also for performance is a culture driven factor.  The organization's culture for engineering practice drives it!

Testing for Performance - Where to Start?

I'm sharing my research work that I'm doing and experimenting on performance engineering and performance tests.  I'm seeing the results and value out of it and so are the stakeholders.

Today, we are getting skilled in exploring and testing without the requirement document and SLAs in hand.  Isn't it?  Haven't you?

I use my MVPT to figure out what are the minimum performance tests for the feature.  As part of this, I will explore with help available aids to evaluate the performance.

To start, I will use these questions to figure out the performance tests:

• What is the minimum viable questioning performance tests that you have got to test this feature?
• What is the minimum viable questioning performance tests that you have got to test this workflow?

Unit Tests for Time and Space Complexity

I will work closely with programmers to gather information on below when the code for the feature is committed as part of Unit Tests.

• The execution time taken by the code of that feature - the Big O Notations for space and time complexity
• Usually the Unit Tests focuses on functional tests and clean code practice
• But, when we test team ask and push for performance data, this can come as part of Unit Tests
• An architect or a principal engineer can set an expectation on
• What should be the time and space complexity of a code for a feature?
• Each functions and blocks need to be evaluated on this
• As said earlier, this depends on a engineering practice culture of an organization
• If the culture wants it, it will be there; else just the functional code will be delivered and not the performance code
• If the time and space complexity analysis outcome is not as expected, the code written has to rethought and refactored
• The review process need to put it back
• The comment with data has to be published
• This will be useful to model the performance tests by test engineers who will be working on performance tests
• Doesn't it look a like a effective useful practice as part of Performance Engineering right in the early stage?
• This is very well applicable to projects running on Agile or Waterfall

Do you have this in your project and Unit Tests written?

The time and space complexity questions should not be confined just to the SDETs [test engineer] interview.  A test engineer has to ask for it and apply it in her or his day-to-day work.

Profiling Tests by Test Engineers

We testers do not get into product's code analysis.  We have to build skill to run the profiling on product's code and analyze the resources data.

• Test Engineers can test the feature's code with the help of IDE's profiling (runtime analysis) and collect the performance data by identifying the performance bottlenecks
• This runtime analysis can profile for
• Memory snapshots
• Thread analysis
• Monitoring resources
• CPU and allocation profiling
• And, more
• The problems and risks can be reported upon analysis
• Compare the two different solution's approach performance data

This information will tell and indicate where is the risk and problem when we deploy the code.  In my opinion, this is a useful information in modeling further performance tests.  This information is first-hand information which is very powerful before we start using any other performance testing strategies and tools to aid the tests.

Get Started with Performance Engineering and Tests

These are available in the IDE.  We think of performance testing tools and ask how to test for performance.  To be precise, we test developers (test engineers) should change our mind and shift for first.  If not, as I say, we will be the bottleneck for first to ourselves.  Did you know this way of testing for performance?  Why not you introduce this in your project and organization?

If seen, these test practices can be used right from the day we commit the feature's code. This is a place to start for the performance tests.   This will be a differentiator together with MVPT and guides the MVPT to design effective performance tests in the context.

I do not say these are best practices and there is no best practices.  But this is a useful practice when the organization and stakeholders ask for it.  Let your organization and stakeholders know how well you can test for performance right from first commit of product's code.

To stop and end here,

1. Just do not test for functionality from day one, also test for the performance from the day one.
2. Influence your organization's engineering culture and developers not just for developing functional code, but, also for the performance code

MVQT: The Testing and Tests with a MVP's Perspectives

I was leading multiple teams and its delivery in a testing service company.  Then, I came up with this thought -- Like MVP, I also have the MVT (Minimum Viable Tests) for a MVP.

Further, I expanded this thought in my day-to-day practice on tailoring to different contexts. I'm observing that it is applying well to the different contexts when I tailor it to the contexts.  After experimenting it for 10 years, I'm sharing this as a blog post.

What is a MVP?

I take this from Eric Ries.  It looks simple and precise to me.

The Minimum Viable Product is that version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort.

I see this technique [and a concept] can be applied to anything when I'm developing.  As a test engineer, I develop the tests and test code in major as part of my testing.  On applying the idea of MVP to my testing and deliveries, I see the value and result.

Reading this blog post of me to know who is a developer.

Testing, Tests, MVP and MVQT

In software test engineering, I see the MVP as Minimum Viable Questioning Tests.

The Minimum Viable 'Q' Tests (MVQT) for a focused area of a feature [or to a feature]

• Helps me to identify the priority tests that should be executed for first
• Allows me to learn information on priority which matters critically to product and stakeholders
• So that a informed decision can be made.

The Q in MVQT stands for "questioning".  I read it as Minimum Viable Questioning Tests.  I see the "Q" as a placeholder for the Quality Criteria.  That is, MVFT means Minimum Viable Questioning Functional Tests to a feature or a workflow.

The MVQT are key to know:

• Have I identified and designed the priority tests?  How do I know that I have got them?
• Did stakeholders get the information which they wanted to know on priority?
• Did MVQT help me to
• Explore and know what I wanted to know about a feature or a workflow?
• How fast I was here to know and learn this?
• How did I develop my tests incrementally?  Did I?  If not, then, is it a MVQT?
• Did MVQT help me to know
• Am I aligned and in sync with expectations of my stakeholders and customers who are using the software product I'm testing and automating?
• Did the MVQT help me 
• In collecting the critical information in a given context for the scope of testing and automation?
• Do the learning and outcome from this MVQT help to reinforce the validated learning of customer?
• Do MVQT result support the outcome of Unit Testing result?

The tests in MVQT has to be consistently revised and evaluated to keep it as a MVQT.  Note this, not all tests are MVQT.  If the number of MVQT is growing to a part of feature or to a feature, it is time to think about what is MVQT for you.

The "minimum" tests are highly effective and it helps me learn and test better technically and socially.

MVQT and Testing

• Sanity or Smoke Tests
• The set of MVQT which helps me learn can the build be taken further testing
• MVFT - Minimum Viable Questioning Functional Tests
• Apply this to a feature or a workflow or to that part which can be evaluated with minimum tests for its functionality
• To update is this aligning to the validated learning of customer [stakeholders]
• MVPT - Minimum Viable Questioning Performance Tests
• MVUT - Minimum Viable Questioning Usability Tests
• MVAT - Minimum Viable Questioning Accessibility Tests
• MVTxT - Minimum Viable Questioning Tester's Experience Tests
• MVST - Minimum Viable Questioning Security Tests
• MVAF - Minimum Viable Questioning Automation to a Feature
• MVLT - Minimum Viable Questioning Localization Tests
• MVUIT - Minimum Viable Questioning UI Tests

You add more of this to your list and context.

In a way, MVQT should ask and look for the testability, automatability and observability.  If this is not happening, then there is no possibility of saying I have got my MVQT.

More importantly, in the CI-CD ecosystem, MVQT pays a major role.  If I should have my tests in the  CI-CD pipeline, then, the MVQT is the way and it focuses on a targeted area to evaluate it.  Else, it is hard, impractical and not possible to test in CI-CD eco system by delivering continuously.

Ask and Review for MVQT

Ask for MVQT, when you review these:

• test strategy, test framing, test design, test ideas, test cases, test plan, test architecture, test engineering, testing center of excellence, and test code

For example,

• What is the minimum viable questioning performance tests that you have got to test this feature?
• What is the minimum viable questioning performance tests that you have got to test this workflow?
• What is the minimum viable questioning security tests that you have got to test this feature?
• What is the minimum viable questioning GUI tests that you have got to test this feature?
• What is the minimum viable questioning contract tests that you have got to test this end point?

Likewise, What is the minimum viable questioning automation tests that you have got to test this feature?

Ask how these tests qualify as MVQT in this context of testing and automation?

This should help you to see how effective is the test strategy in a given context.

Importantly, the MVQT and its effectiveness is a testability to test your tests.

The Credit is to Me

I'm not sure if the idea what I'm saying here in this blog post is practiced by other test engineers.  I have not seen this being discussed about it in public forum.  I have not come across it in my awareness and to the exposure I have put myself.

Hence, I will take this credit to me.  Giving the credit honestly is not a common sight and practice.  I have not got my due credits for using the ideas, thoughts and work that I have come up with.

So, I make it as a open letter and call out that credit for this idea, thought, concept, and practice will be to me when you listen, use and practice it.

The Agile & Testing: We're Pulled & Hidden from "Shift Right"

Do I Understand the Agile?

If I work in a project which follows the Waterfall approach, I still can apply the Agile Manifesto and the 12 principles behind it.

But, am I adaptable to the change and uncertainty for better ways of developing software and helps others do it?  This shows the mindset of me and others involved.  Have you heard this line, "every project that claims Agile, it has [a small] waterfall in it?".

Agile Software Development is not wrong!  It has values in what it says and for what it stands.  I have no doubts in it!  It describes how I perform a activity and respond to a situation in the software development.

Frameworks and Methodology

We have picked the frameworks which are under the Agile's umbrella. We pick the methodology that suits our work and have customized it to our contexts.

This is what Agile also say, the team will always need to adapt its use of a framework to fit properly in its context.  

Dr. Alistair Cockburn, says, the Agile methodologies are the conventions that a team choses to follow in a way that follows Agile values and principles.

The Agile is Not a Problem!

We do not collaborate well in solving the problem and say Agile is not working and creating more problems.

In reality, how we respond to the uncertainties and what got prioritized, it will derail us.  We derail from what we should be achieving as team [and as an individual] in a given time frame and context.

The Agile as I Understand

In a nutshell, the Agile is about

• Balancing effectively with collaboration in the context we are.
• Identifying, learning and solving the problem as a team no matter where we are in the project's timeline.
• Being ready to face the uncertainty and responding to the changes.
• See the progress and value in what we do as a team and bring the "continuous" in practice.
• Having the consistency in the excellence of our work and the value we deliver with it.

Read the Agile Manifesto and its 12 principles.  It is the day-to-day life of a team who works to build a software in a continuous delivery having the values to a stakeholder.

Testing and the Agile

Testing is a collaborative activity.

The Agile also talks about collaboration as an essence and necessity in the continuous delivery of the software we develop.  

The Agile talks about short interval consistent deliveries.

• How should I fit myself as a Test Engineer here?
• How should I upskill myself to deliver in short intervals as part of continuous delivery?  

I ask and say to my test teams,

• Think of delivering value consistently in the shorter intervals.
• Rather saying -- releases in short intervals, call it out as a value to a stakeholder in regular short intervals as a continuous delivery
• It sounds positive, affirmative and profound.
• Upskill and help your peers to build the skills. List the skills, let us discuss and start our practice on it.
• This mindset changes the perception in how you approach the testing.
• In these short intervals,
• Let us prevent bugs in the ways we can think of.
• Finding bugs also need resources.
• Let's start investing in building and improvising the resources to prevent bugs.
• How can we prevent and where to begin?
• For what should we test and automate other than functionality in these short intervals?
• What is the priority?
• If I do not test and automate for evaluating any particular quality criteria, does it impact the value to a customer in what we deliver?
• If I do not test and automate on a particular seam and system's boundary interface, does it impact the value to a customer in what we deliver?
• How am I collaborating with stakeholders?
• Who are my stakeholders?

In short, I see the Software Test Engineering practice in the Agile setup as this,

• We [team] exploring technically and analytically;
• Evaluating collaboratively in the software development and engineering, by having the effective staged feedback loops in place to improvise the delivery;
• By responding actively to the changes and uncertainties that comes anytime in the timeline of a delivery and practice;
• Thus, the consistent continuous delivery is delivered with value to the stakeholders.

Note: This is my observations so far having worked with 46+ startups in my software test engineering practice.

We're Pulled and Hidden From - Shift Right

I and you are said to Shift Left and start the testing from the inception of a feature's discussion.  And, further we're said, we have to shift left from right.

Did the Agile manifesto and its 12 principles say explicitly to Shift Left from Right?  Who said that?

What I see, How I see and understand the Right and Left here, matters a lot!  If I see the Right within the delivery loop, I'm setting a trap to myself to be late and lag back.  And, probably this trap gets set in the work, that is, to see the Right in the [continuous] delivery loop.  At certain context in continuous delivery loop, starting from Right is a need and necessity.

I see the Right outside the [continuous] delivery loop, as well.  In this Right, I see:

• What is coming up in the tech stack and technology?
• How this is a need?
• How this [is to be and] will be adapted by tech solutions we deliver?
• How should I upskill myself here so that I fit into the loop of continuous delivery?
• In short, the Right should be observing the trend and its analysis so that I'm aligned being agile.
• I will be skilled to test and automate in this change and uncertainties.

The Agile never said to completely move or step out of the Right.  Or, I have not read it in its manifesto and principles. 

You see and learn, what is your Right and ask am I aligning to it for being agile in the Agile.  After all, the Agile says adapting to the changes and uncertainties, yet delivering the value to the stakeholders consistently in short intervals.

If seen closely, the Agile is also about balancing, isn't it?  If I'm moving towards Left and me standing in the Left, am I balanced?

In short, we are being pulled and hidden from Shift Right.

Shifting Right is important to see what is outside the delivery loop.   Are you seeing your Right?  If yes, what is the trend analysis you see?  Where you need to upskill?

Shift Right can also be to look what you can bring into your Shift Left for better development and delivery.  For example, what you see in the market and business space for your product?  Do I have anything to add now in Left from this learning?  Do you see that?  Are not we missing this when we pulled and hidden from Shift Right?

Left or Right, if I do not respond, adapt and deliver for the changes, I need to ask, -- Am I agile and testing in the Agile?

To conclude, know what is your Shift Right while you are delivering values in your Shift Left.  Strike the balance and ace.

My Work, My Fit, and Company's Goals

 

I, My Role and Expectations

At least once a day, it is useful to think about oneself.  I started doing this late in my life and career. I started doing it in recent years.  If I do not think about myself, I will be lost very soon.  This is not selfishness; it is self-care, which is what I'm learning.

It is essential for me to think about myself, because:

• It helps me to see what I'm
• It helps me to see where I moved today
• Does this move help me personally?
• Does this move help me professionally?
• What benefits does it bring me?
• What benefits does it bring to those with whom I associate and work together?
• Does it keep me in sound mental and physical health?
• Did I learn today?
• Something new?
• Anything I refined and unlearned?
• Does it bring any costs and cons to me?
• Am I fit?
• Fit to where?
• Fit to what?
• Fit? How?
• Fit? Why?
• Fit? When

In all the roles I take in my personal and professional life, I'm evaluated at some point in time.  I will be judged for:

• Did I fit?
• Did I do my role
• Did I meet the expectation?

The problem is not that I'm evaluated.

The problem is I'm evaluated without saying what makes me fit to be in the association and how I will have to meet the expectation.  Some associations can remove us while some cannot.

When I say this, I want to say this -- the word family is often misunderstood; not all associations can be a family although the word family is used often in associations.  This is reality and not a fact!

Does family eliminate me if I do not fit in?  I don't know!  At least the hope is, family is where I can be myself; without the thought of me being judged and evaluated for what I take and bring to the family.  My family as well have expectations from me in the different roles I live in with them.

When I can see this in my family, why don't I see this in the place where I work together with other people?

Do I fit in here for what I make out of this place (company) and take it to my family, home, and my life?

I wish my home and school had helped me learn this question early in my life!  I expect it now because I realize the "value of fit", now, that is, after I graduated and started to work with people in the organizations.

I consistently learn that every one of us is replaceable in any association, be it family or a workplace.  And, it moves on; it does not stop.  If not replaceable, it is manageable to continue and move on.

When we are in the association, how fit we are so that it is hard to replace?  Maybe that's a price (value) tag and a necessity of one!

The Response, That I Should Evaluate

As a responsible colleague and team member, I promote the discussion of this question at least once a month.  I ask this question to whom I report at work.

I will have this question in every one-on-one catch-up that I will have with my reporting manager.  And, I expect a response to this question and want it recorded for future reference.

What is that question?

How does my work fit in with the company's goals?

Evaluating the response:

• How do I evaluate the response to this question?
• What should I do on the evaluation of the response?
• Why should I evaluate the response?
• What should be my next course of action?
• After all, what is my response to this question and how do I evaluate it?

To get promoted to the next roles,

• I need to be solving the problem of my higher (or next) role
• I need to have the capability (skills) to solve problems of my higher role

But this is not a question of promotion.  It is the question of being fit for the company's goals.

While I get promoted or to be promoted, my work may still not fit with the company's goals.  Identifying this early helps.

I have learned, sometimes the promotion does not necessarily come with the fit for the company's goals.  But then eventually the fit will be evaluated at one stage by someone in the company together with a promotion given.

This has led me to ask this question consistently and then evaluate the response with the business, political and rational mindsets.

I say the same to my team, that is, ask this question for yourself and to the reporting manager.  Evaluate the response that comes to you.

Should you ask this question to your reporting manager in each month's one-on-one catch-up?

The Fit Equation Changes

In the team and company, we believe:

• We are contributing
• We are a value-adding fit type
  

We keep saying to ourselves how we make a great fit and difference.  Isn't it?

This "fit equation" keeps changing every day or quarter or year or appraisal cycle.  I learn, this "fit equation" keeps changing rather than evolving.

Adapting to this consistent change and delivering is evolving.  This is my understanding for today.

Biases, Communication, and Problem Solving

We all are in biased mindsets and perceptions at any point in time.  The people in the company need help to break these mindsets so that one's fit equation is questioned and assessed regularly.  In my opinion, this is a great assistance that a manager and a leader can give to her/his people.

I expect the managers and leaders to ask the company:

• What the company wants from the people?

We people in the company and in the team, let us ask the manager and leaders:

• What the company expects from me?
• What is my fit equation?
• Does the current work that I'm delivering fits the company's goals?

I have heard most times from people saying, "I was said that I did not fit with the company's goal".

How will one know what is the company's goals and how can one align with them unless it is communicated and recorded professionally?  I see, to start it needs communication, clarity, and affirmation first from both ends.

Does this solve the problem?  No!  It gives an onset to understand the problem and the differences to fix.  With this, the manager and leader can help the team, and vice versa, in solving the problem.  Thereby contributing to the company's goals by aligning with them.

If you are a manager or a leader, make sure you have this as an essential practice in monthly catch-up to assess this fit equation and let know your people.  I love seeing this initiative from managers and leaders.

This is one of the leader's fitness to be in the role to assist people and the company.  By doing so, we will help the company, business, employees, investors, and customers.

To reset this post's intent equation:

• How the work expected from me fit in with the company's goals?
• How does the work I'm doing fit in with the company's goals?

A Direction Sign for Beginners and Everyone to Start and Grow in Software Testing

 

I'm volunteering for the Agile Testing Alliance (ATA) for this year 2022.  This 10th September 2022, we had an ATA Round Table Talk 2 on account of International Testers Day.  We planned two events on this day -- QuizATAhon-1 and Ask Me Anything (AMA) with ATA volunteers.

One of the questions that we volunteers got from the community is:

"Best advice for beginners who wants to start there career in the testing field as per there experience level"

I shared my thoughts on this question in the AMA session.  I have a strong intuitive feeling that this will be the question of the interns, freshers, and also of experienced test engineers.  Hence, I want to write it as a blog post, so that it can be referred to as one of the directional heuristics.

This is Not an Advice

What I'm sharing in this post it is not advice.  It is information. You can use this information to advise yourself on what you should be doing to grow consistently in the practice of Software Testing and testing job.  As said, it is a directional heuristic that you can use.

This information will make sense to every one of us irrespective of experience in the industry with software testing as a career and practice.  There is no best advice.  I see, advice is information or carries a piece of information that can show us a direction.  And, the advice is a heuristic!

Direction Sign To Be a Skilled Tester

This will be super useful to you if you actually make use of it to your need and to the maximum possible extent that you can.

1. Find your Software Testing Community
• You might find the community quicker than finding the people, books, and syllabus of Software Testing
• If you don't read Software Testing and Programming resources, someone in the community might be reading it
• You can use their learning for your growth
• Look out for the Software Testing Communities in your place (city or country)
• Just do not look at one testing community; find more than one active communities
• Collaborate with the communities
• Interact with fellow Test Engineers in the community
• Start contributing to the communities in the possible ways you can
• Note that, the groups on Social Media will not make it a community
• People come together in a community
• Events, Meetups, Conferences, Discussions and much more will happen in the communities
• Today we can associate with Software Testing Communities which are in other countries
• Could be this community will have one of its chapters in your city or country
• Find the Software Testing communities
2. Mindset - Do not get easily influenced
• You are bound to get influenced and follow easily
• when you look into the community and the people in the community
• with their works, writing, accomplishments, social presence, and identity
• when your fellow peers in college or the workplace talk about testing to you in anyways and in any dimensions
• It is their opinion
• It is their understanding
• It is their assumptions
• It is their mindset
• and, with what you read or watch about testing posted on social spaces
• Have the mind and thought that questions; try to seek what makes sense rationally to you
• Be practical and experimental
• Do not attach your past and present experience's emotion to others' opinions
• You evaluate it
• Pick what makes sense to you and to your context
• Do not be a mind of other Test Engineer or practitioner
• Have our own voice and identity
• Express your opinion while you respect the work and opinion of other Test Engineers and practitioners
• Develop your working style, learning style, and problem-solving style
• But do not stop observing others and how they are doing it!
• Question!
• It is about being included than being influenced
• Do not stop questioning
• Talk, and talk with respect!
• A test is a question asked to learn what it is and to understand what actually it is
• It needs a mindset
• A mindset not to get easily influenced and accept an opinion or a thought
• Note: Do not be influenced by this blog of me; just read it as a heuristic
• You build your approach using this heuristic only if it helps you
3. Knowing the chaos around Software Testing is very important
• Everyone knows Software Testing and everyone believes to have an idea of what it is
• This is the state of mind in the college that teaches computers and programming
• This is the state of mind in the software industry
• Chaos starts right from the syllabus, textbooks, and reference books, and with you
• Here are some chaos and myths existing for 20+ years and they will exist in the coming years
• Software Tester has no bright career as Software Testing is not technical
• End of Software Testing is coming soon
• Software Testing has seen its end
• No need of knowing the programming
• It is a necessity in today's Software Development context
• It is an easy job
• Is it so?
• Try finding at least 3 business stopping risks and problems in the system every day
• If you do this consistently, label it as an easy job
• Let your business stakeholder acknowledge the findings of your testing as a business stopping risks and problems
• No good money earned when compared to a programmer
• No career in Software Testing as we grow with our experience in the industry
• To be a Software Tester one need not be technical
• It is a Women's job
• Yes, that is how it was projected and was said
• I'm saying this from my experience in India
• I'm not aware of how it was or it is in other countries
• 2014 and onwards, I have not heard this statement
• Happy, that we don't hear this today
• Thanks to SDET and SET roles
• At least, this removed the gender bias is what I see
• Today both women and men work in the role of SDETs
• I worked with women SDETs and I did learn from them; I say this is in pride
• I'm happy that women are taken out of this framing today
• I'm very happy!
• We don't have to bring gender discrimination in Software Testing career and jobs anymore
• I see, there is no job that women cannot do today in the technology space
• This is an empowerment of women
• Women are equally skilled and work to upskill
• We have skilled women technologists, CTOs, VP Engineering, Architects, Test Architects, SDETs, & Test Engineers
• I'm happy that I have worked with technically skilled women and I reported to a few of them
• Ah! I should stop calling men and women here; it is we Test Engineers
• Do not panic with the chaos that is created frequently and consistently
• Talk to your community
4. Find your Mentors
• I said mentors and not a mentor
• Have more than one mentor
• Know how to work and practice while you have more than one mentors
• Respect your mentors by giving the credits and with your gratitude
• Find your mentors in the workplace and in the community
• Good if your mentors have the contrasting thought process and ideologies
• Pick from both sides
• Know what both sides advocate and practices
• Apply the appropriate one to your work when the context demands it
• Do not be the mouthpiece of your mentor
• Do not be the mind copy or replica of your mentor
• Do not imitate your mentor
• Seek your mentor's assistance in you being you
• Seek help in growing with your identity and voice
• Have a voice and your identity
5. Communicate with your Software Testing Community
• When you talk to the community, you will know
• where you are
• where is Software Testing
• the challenges your fellow testers in the community are having
• how the challenges and problems are being solved
• It will set you a tempo with an attitude if you talk and share your learning
• In simple, just read or listen to the problems the testers share everyday
• This is awareness!
• At least, it opens to the awareness of technology, tech stacks, problems, solutions, industry, people, and more
• Attend the meetups
• Small crowd and high interaction with networking
• High exchange of learning and experiences
6. Be the Technical Mind
• We Test Engineers are said to think like a user
• Good!
• This is an empathy mindset that is being cultivated in us for the users who use the solution we are building
• But this empathy is alone enough today? No!
• Empathy with technical skills will be much value adding
• And it is today's need
• When I say technical, it is not just programming
• Programming is one small aspect of being technical
• Let your technical journey start from learning:
• How this works and what makes it work
• Know the technology layers internally and externally when you learn how it works or when it does not work
• Start here! And expand your technical capability dimensions
• Today, we are said and expected to think like an engineer
• An engineer who understands the user
• An engineer who understands the business
• An engineer who understands the investor
• An engineer who understands the management and managers
• An engineer who understands all this can change at any time
• An engineer who can work, scale and deliver in a startup mindset and environment
7. Seek and Share Awareness
• What sets back we Test Engineers is the lack of awareness before lack of attempts to practice
• We don't work, collaborate and associate to be aware of
• We are not aware of -- what to be aware of
• This includes me as well
• I consistently keep trying to be aware
• Do not just attend  and be part of the Software Testing community
• Attend the community of programmers, DevOps, products, businesses, startups, enterprise
• You will see the direction sign -- which happens to be a heuristic
8. Know what is your Software Testing
• Everyone has their understanding of Software Testing
• Engineering Managers and Director of Engineering with whom I worked had their own understanding of Software Testing
• No one was similar nor did match anywhere
• All have their version of testing in their work
• Know what is Software Testing you want to practice
• Software Testing is contextual
• Tailor and deliver Software Testing to what is expected at your workplace
• While you do that, try educating your fellow Test Engineers
• That's the seeding
• This is the place where we can seed and harvest for a better tomorrow in Software Testing and for our growth
9. Practice your Software Testing in all dimensions
• Software Testing is with people, software, hardware, and more
• When we are dealing with Software we cannot be away from programming
• Programming gives a different order to our testing
• Practice programming
• Embrace it
• Shell out the fear of programming
• Start small; more importantly start and continue
• To keep going, find one tester in the community who wants to do it
• Go to your mentors and follow up with your accomplishments, setbacks, challenges, and problems
• Get unblocked; make progress; learn, implement and share
• Do not run away from the practice of automation
• It is a necessity for today's Software Development & Engineering
• Find all possible dimensions you can
10. Build your portfolio
• Explore how to build your portfolio in Software Testing
• Ask community
• We have several portfolios to refer to in the community
• Build it and continue improvising it with the time and context
11. Solve the Software Testing problems
• Before solving at least be aware of the problems
• Community is one place where you can hear the problems and challenges
• Contribute to the solution and solving
• This helps you a lot as a practicing Test Engineer
• Problems come in flavors and contexts
• Testing, Automation, DevOps, Product, Requirement, Delivery, and more
• Have yours hands-on on all these verticals to help and scale your testing and automation
• This will make you talk and help you grow with experience and learning
12. Tune up yourself with Business, Political, and Management skills
• Not all problems can be solved with a technical mindset and skills
• It needs the skill of political and management orientation
• Do good and be good
• Work on how you communicate; it is important
• Know how the business decisions are made
• Know how business decisions influence technology, engineering, and anything
• The business decision need not be based on logical and technical analysis
13. Beat your EGO and respect your self-respect with dignity for Software Testing
• Ego has killed a lot of us from talking and growing
• Ego has made many managers lose their engineers
• Ego had made many testers lose what their managers could offer to learn and grow
• Software Testing has its touch points in every vertical of Software Development
• Programming
• DevOps
• Product Management
• Solutions
• and, what else?
• Testing is done in all these verticals
• Ego can set us back and block from what we can make for our benefit and contribute to the organization and business
• Managing the ego is a skill
• While managing ego, balancing one's self-respect is a challenge
• Know what is the objective and what you make out of it for your growth
• While doing this keep the dignity of Software Testing and team morale high along with your self-respect
• Talk to your mentors!
• Yes, talk to your mentors
• Mentors might be handling ego, self-respect, and dignity of their work and practice
• Your mentors can tell you the ways to do it
• Take care of your Mental Health and help others too
• Physical health is also important

This information should hint you with the direction of practice, growth, and excellence as a Software Testing practitioner.  I'm here to connect with you anytime and talk more and take it forward.

Bug - The One Word and Mulitple Incorrect Alternatives!?

 

What is Your Word?

I love to meet and collaborate with testers and learn what they are doing and how they are practicing Software Testing, Automation, and solving problems.  When I collaborate, I see testers using different words for communicating what we do.  For example, one will use the word "bug" and few others will use the words "issue", "defect", etc.  What word do you use when you talk about the bug and information you find with help of your testing?

The Word and Alternatives Used

These words are common in the Software Development team when referring to a bug:

• Anamoly: which means abnormality
• Defect: a legal perspective for the unexpected or not wanted or false promise
• Error: a condition due to user input or/and program execution
• Issue: a concern
• Problem: the difference between expected and reality (actual)
• Risk: a potential problem in near time
• Bug: ?

Examples:

• The abnormal growth of bone
• The defective electronic equipment that is used where people are involved
• Error on saving a file with no name
• The team has an issue with the time available
• I have a problem in sharing this report with the customer
• If we do not publish the app today we risk losing users in this season sale
• What's bugging you these days?

If observed, to an extent all the above words have something in common.  That is -- relationship, loss, cost, experience, emotions, expectations, compared to something and evaluated, feeling, and more.

But, that does not mean I can use them casually and interchangeably.  The one thing that is not so common to all is CONTEXT.  All of these have their own context and are different.

Is it a welcoming practice to equate and assign the word "bug" to all the above words?

Bug - Annoyance to whom?

The people and practitioners have different definitions tagged to the word -- Bug.  One word which looks non-technical and asks for attention in addressing it on evaluation is -- annoyance.  I won't bug you anymore!

The bug can be related to annoyance for one who has a relationship with the product or service or company.  The bug is the outcome of my experience when I look for what I need.  The bug is also the relationship that I carry with the product or service that I use.  Since I have a relationship, I feel the experience of annoyance.

I see the bug is not a word of tester or programmer or product owner or anyone in the software development team.  We use the word "bug" to communicate and document the annoyance that a user experiences in using the service we provide.  To extend the view, I can also include the business here.  If it annoys the business in delivering, giving, and making the benefits, it is a bug.  What do you say?

If my annoyance is also the same as the annoyance of an actual user and business, it carries a weightage and importance in the team.  When I say the bug report, we talk on behalf of the product, user, company that is offering service, and the team building the software.

To Sum Up the Word

It is not termed as Anamoly Advocacy, or Defect Advocacy, or Issue Advocacy, or Problem Advocacy, or Risk Advocacy.

It is termed "Bug Advocacy"!

We are trying to avoid or prevent:

• A defective product in the market that can be sued
• A problem with loss and cost for using a product or service
• A problem to business, and
• A user to be away from the [risk of] cost and annoyance for using product or service

It is up to one as a practitioner to use the appropriate word to context.  And, it must start from we Test Engineers / SDETs.  

What word to use is your choice now!  Bug?

Blog Post Update - 23rd August 2021

ISRO declared the mission of GSLV-F10 launch could not be accomplished as intended due to the technical anomaly in Cryogenic upper stage ignition.  Refer to the below image which is a tweet from ISRO mentioning the same.

Why is the word "technical anomaly" used?  And, not a bug, issue, error, or defect?  Instead, the word "anomaly" is used?  

If the word anomaly is considered as

• irregular
• not expected
• deviation from the expected
• abnormal
• something different
• deviation from commonly accepted rules or readings, and
• more...

If closely observed,

• It is a consistent output or outcome that is observed for a time period for a set of input and factors
• Any change in this outcome or output, is treated as a peculiar case?
• Or as abnormal?
• Or as deviation from what is usually observed and recorded?

Why don't we consider this as a bug, then?

• When spoken about the bug
• In the first impression, it is the emotions and experience captured or heard, or recorded
• Necessarily the root cause analysis or initial debug is not done at the time of calling out an experience as a bug

Whereas spoken about anomoly, I learn

• A set of analysis and experiments are done and the outcome is recorded
• This outcome might have shown the consistent behavior for a set of input and factors
• Anything deviation from this outcome for the same set of input and factors is considered as an anomaly?!
• Though it is unexpected, the context i.e. pre-run tests and rational experiments do not let it be treated and term it as a bug in the communication
• Instead, the word anomaly is used in communication

2020: It Is Costliest If Not Shipped Quickly - Fixing few bugs later is okay!

When you read this post, I assume you are working in a team or organization which has imbibed Agile and its various approaches into practice.  I see it is more mindset and culture, than just saying it as a practice.  I have worked in team and projects which had Waterfall approach. I'm working with teams and projects which claims to be Agile in approach.

Here is my tweet and the question of @J_teapot.  His tweet is interesting and striking to me.  Hence, I write this post.  Thanks, @J_teapot for asking.  I enjoyed that question of yours.

Days of Waterfall

It was in early 2000s (to be precise 2005-2007), I worked in project which had customized waterfall approach.  The build to testing were delivered in iteration and not at the end.  In the course of project, it got adapted to Agile practices and as I remember it was around 2007-2008.  Yet the shipment of product being developed happened once in 3 or 4 months.  This shipment time remained as is Waterfall and in Agile.  If asked, why that slow in shipment when Agile, it was business decision.

In my initial days of career, I read this in text books of Software Engineering and Software Testing -- "Bugs are more costly to fix when found later."  The same was said by managers to we programmers and testers.  Those were the days where no much blogs and posts in web on Software Testing, Software Development and Practices.  Google, had just come in few years back, then.  It was the period where Software was getting into every spaces of business and daily life.  The pace of delivering software to various business space picked up its pace.  The technology platforms emerged and so the practices.

Days of Startup Era

In the era of technology start-up that is around 2012 and onwards, the shipment pace for "working" piece of software took cut from months to a month and to a week.  Then the era of programmers led technology start-up came in.  The idea of MVP (minimum viable product) and pitching to investors for series of funds, all came into the news.  If you notice, software was developed earlier and, in this date, as well.  The difference is the time and how it shipped, was the business call in major than being a tech call.  I see this is appropriate to remain in business, competition and for being relevant to needs of customer and market.  That change has to come in quick pace.

In 2020, this is no different.  In fact, the shipment pace has got much quicker and it goes in a sprint of 14 days (including non-working days).  You can refer to the train model of backlogs in a sprint for more details.  If did not get deployed and rolled out for install, business will be impacted as earlier.  Who answers for the investors?

The approach of software development has evolved and so the testing also has to adapt.  If had a "working" piece of deployment and serving the customers with value claimed, the team will be relieved; else, the trouble starts for team.  I have been here and I know it.  The MVP which claims to deliver the mentioned value, has to be delivered.  Whatever one builds or adds to MVP, should continue to deliver value claimed.  If there are bugs in shipment that can be bearable, then still fine unless it blocks the value claimed.

I have been in state, where the value claimed by business and MVP, not delivered after shipment and deployment.  The are several reasons as multiple works to ship one working software system.  Fixing that in few minutes and deploying it is expected besides the cost incurred out of it.  It so happens, the deployment and roll out goes while the testing is in progress or not yet started for that version of software.  In a way, it is a strategy and also a business call.

Ship it first - Mindset

In 2020, it is costliest if not shipped quickly; fixing few bugs later is okay.  That's a business call and a strategy.  We have to be there on time with service catered.  In my opinion, unless the value claimed to offer by MVP and product as a whole, is being delivered, all is in control despite the bugs in production.  That said, I'm not ignorant of bugs in production.  I will continue my work in finding them.  I have evolved to adapt my testing and skills to need of business, market and software system we build.

If asked what is right that is shipment once in months or once in a sprint (2 weeks, usually), both are right.  I hear few teams still ships once in 6 months today as well.  I work with teams which ships in a week and in two weeks.  Not to mention, I have worked where the requirement comes from business and shipment happens in few hours.  It is all about, where I'm here.  That as well defines the time available for testing.  I'm not sure what today's Software Engineering and Software Testing books say about fixing bug later.  Fixing later is not a habit but a situation needs today when it comes to business.  I wish, when a testers being technical and understands the business side, it adds value much more to the testing, team, business, organization, customers and to the product.

To summarize, it is business and market demand that has kept the shipment in a sprint.  Prioritizing the blockers (and bugs -- know & unknown) for shipment is a skill which a tech (programmers & testers) team has to groom in my opinion.  I see this is adapted in service industry and as well in the tech product organizations.

What Structure Does a Test Have?

I was an audience in webinar from Manju Maheswar on Heuristics.  This webinar led me to discussion with Klára Jánová on how the heuristic will help in having structure for problem solving.  Here are my tweets as my discussion.

There is a question from Klára Jánová -- "Thanks! What structure does a test have?"  In my opinion this question goes down to fundamental and philosophical level of Testing.  When spoken in casual, the outcome of a test is seen in binary that is pass or fail.  Is that right or not so right, that's altogether a different topic of discussion.  I will not get into it.  But that binary is associated as result to a test and it has an experience attached to it.  That "experience" attached to the test is an "essence of test" which I would like to bring here in this post.  In simple, if a bug is an experience that I encountered in using the product, then test is an experiment to know what is that experience.  The test exposes a tester or anyone to an experience with information, as an outcome.  How we act upon this experience and information on witnessing it, is what tells the further story.

What we feel out of an experience is the shape or structure we give to it.  That said, the test has a structure to it as an experience. This experience will let us to respond further rationally in a structured and organized way to learn further by debugging.

Apart from the said above, there are much more elements that adds structure to the thought of a test identified. The picture shared below will give the gist of elements which fine tunes a test to be precise, deterministic, practical and an experiment with a question.

Above all these, a test is a heuristic.  That means, an experience is as well a heuristic.  So, the software is a heuristic having sets of experiences to its users.  The software has a structure in multiple forms.  The functions (methods), classes, packages (modules) and the data structures used gives the structure internally to a software.  How the product is built and interfaced gives the external structure.

Now, if I happen to talk in this philosophical tone may be not all take it seriously, is what I believe.  I can understand it and nothing wrong in it.  That’s an experience too!   I will have to communicate and talk how the team with which I work communicates, you see that’s a context.  Usually the interests will be in -- binary that is pass or fail; the artifacts which is by-product of testing activity (test cases, bug reports, etc.); the tools and more.  I will have to work in that mode in those contexts.  A function (method) and class written will have yield to an experience from what it does.  Yet I hope there will someone in programmer, manager and in software engineering, can relate to experience part of a test.  When I talk to testing practitioners who speaks the language as this, I talk to them as this.  That's the context of people and practices, again.

Okay, how's the experiences are structuring for you from the encounters with the tests you executed and from the code you wrote?

Test Charter and the Four Components of a Charter

A tweet from Ministry of Testing was -- Do you use Test Charters?

Speaking out of my so far practice, I heard the word 'charter' 10 years back when I was referring to SBTM in James Bach website.  Till then I had not heard the word -- charter.  The two words which confused me then was -- Mission and Charter.  

I looked into the sample SBTM reports which James Bach had shared.  I observed the word 'mission' for each session.  That is each session is chartered with a mission to accomplish.  Meanwhile, I strongly believe the spark that took off to testing globe wide referring such testing resources had its birth in Bengaluru then.  Most will say nah and disagree to it.  But that's the truth and Weekend Testing was one of its outcome.

Coming back to the subject of charter, I say, it was not that easy then for me to write one.  I got better each day and I try to get better today as well.  While I worked in the role of Test Coach in Moolya Software Testing, I pair tested with freshers and lateral hires.  I insisted the testers (whom I was assisting to practice testing) to write charter on their own.  I could see the difficulty they were going through and it was a mirror of me showing what I went through some time back.  In beginning all had difficulty to practice in session as they could not concentrate for 30 minutes (forget 45 or 60 min or 90 min) and to take make test notes in parallel.  Yet, there was no practice without a session and a charter to it.

To keep it simple and make lives of the tester easier, I broke the charter into four components. They are:

1. Intent  -- What is the intent of my testing in this session?

2. Target -- What I should be testing?

3. Resources -- What I should be referring while I test?

4. Information -- What information I have to learn from the tests and how to report it.

For example, say the charter of a session is -- Browse through the different doctors displayed based on specialty by swiping the doctor info card in the app. Validate if the subsequent result pages of doctors displayed is still relevant to the specialty chosen, while a doctor can have multiple specialty.  Refer to the HTTP request and response coming in for each result page and validate data displayed on client end. Report with logs and test data when you find or feel there is a problem.

The same charter can be written as this -- Test for the specialty displayed for doctor in search result is consistent in each result pages returned by server. Make sure the test data of doctor have multiple specialty. Report with logs and test data when you find or feel there is a problem. Make note of the confusion if you have any in analyzing the search result.

Also it can be written as this -- Test for the doctor card shown in search results based on specialty chosen.  Report with logs and test data when you find or feel there is a problem.  Make not of the confusion if you have any in analyzing the search result.

If I had to break this charter into four components, then:

1. Intent -- Search for doctors based on a specialty

2. Target -- Doctors info card displayed in the app that is search result pages

3. Resources -- Specialty of a doctor; HTTP request and response; Data displayed on client app

4. Information -- The search results returned and displayed are consistent with chosen specialty? Report if any problem and confusion.

All the above written three charters mean the same.  But how it is written and details provided, varies.  What style to pick and details to provide it depends on multiple reasons when I work with team.  When I work as a only tester testing, I keep the charter pretty self explanatory with details to the reader of session notes.

In my opinion, a tester gets better in writing charter when these four components is clear and can be identified. 

Backward Compatibility - What and who drives the compatibility?

A tweet of Ministry of Testing read as this -- How do you decide how backwards compatible your product should be?

I learn, there is no general rule and as well no specific rule that applies when it comes to backwards compatibility, forward compatibility and the compatibility.  It is highly specific to what product we are dealing with and how the system around it are developed, or being developed or will be developed.  Also how it is deployed.

While compatibility is one of the main feature of product feature, it also signifies -- the design, the deployment style & patterns, communication and the exchange of data.

Further to be precise, compatibility is context specific task.  I have not come across where it is generic task where it is applied and applicable to any where.

If asked what software can come into compatibility spectrum, then:

>> Database System

>> Server

>> Client libraries like SDKs

>> Server Configurations

>> Operating System

>> Hardware Configurations

>> Client application versions being used by user and number of users

>> Server configuration versions being used in different client applications

>> Browser versions, it was a major compatibility concern a decade back

>> Application system version and its dependencies compatibility version

Few analytics that influences the decision on compatibility

>> Analytics showing the user on specific versions of libraries

>> Analytics showing the user on specific versions of OS

>> Analytics showing the user on specific version of hardware configurations

>> Analytics showing the user using specific feature which involves server configurations

>> Analytics showing the conversion funnel on specific setup of system

Few emotion that influences the decision on compatibility

>> Experience of user using a product on specific setup and configuration

>> Customer Support analytics on problems, queries and user satisfaction on specific setup and configuration

>> History of previous compatibility and migration

>> Competitor's presence

>> Team's opinion

Business aspects

>> Negotiation extent

>> Bearable Cost and Non-bearable cost

>> Business Value

>> Social Value

>> Strategic Value

>> All above converting to business profits

>> Survey carried out

>> Timeline

>> Budget and Resources

>> Talents -- people

Environment

>> That simulates the backward version of the system

>> That simulates the provisioned version of the system

>> Outcome of the assessment in such environments

If seen, where do tester fit here to decide the backward compatibility of the product?  In my experience, the decision of backward compatibility is most times strategic from aspect of the technicality.  In the enterprise ecosystem, the same strategic will weigh more on the business decisions and then the technical decisions.

The executing tester most time will not be involved here in such decisions.  Could be a person leading the testing team or the engineering efforts can be involved in such meetings.  In start-up environment I have worked so far, the decision meeting most times did not have the testers involved.  However, on getting the heads up for backward compatibility need, I use to assess the risks in system and its sub-systems.  Then, I shared and had a meeting scheduled if there was a need of different stakeholders and teams to clarify and resolve any risks mentioned.

In simple, the business don't want the service to be broken for a user.  The user should get the service that business provides on using the software application.  Further, the conversion what the business expects has to be what the business expects.  If not, then there is a problem to user and as well to business.