Waterfall or Agile: Testing for Performance - Where to Start?

 

Do you understand the Agile?  I have shared my understanding here; give it a read.

The eighth question from season two of 100 Days of Skilled Testing is:

Can you share some best practices for conducting performance tests within an Agile development environment?

Best Practices and the Agile

The irony is, the Agile says, there is no best practice.  It asks, to tailor and fit the practice to the context so the continuous delivery and value is delivered consistently upholding the Agile's principles.  

Yet, we talk about the best practices in the Agile's context, like the eighth question asked here.

What is the effective way to test in the continuous delivery?

As a test engineer, how can I start thinking and testing for performance from the inception of a feature's thought?  I see, it is not hard to do so.  As you read further in this post, you will have a perspective and awareness to do it.

Performance in Waterfall and Agile

I learn, the performance is an experience.  It does not differ because of the Waterfall or Agile.  If the performance is not a pleasing experience, it will impact stakeholders no matter it is Waterfall or Agile.

But, the question when evaluating for the performance is -- where to start, when to start, how to start, and with what to start?

As of today, I do not see differences in the mindset and skills one has to have for testing of performance in Waterfall and Agile.  Could be the approach differs in certain phases here; otherwise, I see the same in both practices.

I will rephrase the eighth question to this:

What is your practice to evaluate the performance right from the start of product development in your project?

I do not want to wait until to hear -- the development is completed and deployed; now we can start running the performance tests.

What can I do as part of performance tests from the first day of development and first commit?  This is my intent and area to look in strategizing the testing and tests.

The Culture of Engineering

At the start and end of the day, when we developers start and finishes the work,

• How the work is done and why, is defined by the engineering culture practiced by that organization.
• The Performance Engineering of the software products and solution being built will be driven the by the culture practiced.

The Test Engineering and how we test and automate will be driven by the culture of engineering practiced in the organization.

Writing the code not just for building the functionality, but, also for performance is a culture driven factor.  The organization's culture for engineering practice drives it!

Testing for Performance - Where to Start?

I'm sharing my research work that I'm doing and experimenting on performance engineering and performance tests.  I'm seeing the results and value out of it and so are the stakeholders.

Today, we are getting skilled in exploring and testing without the requirement document and SLAs in hand.  Isn't it?  Haven't you?

I use my MVPT to figure out what are the minimum performance tests for the feature.  As part of this, I will explore with help available aids to evaluate the performance.

To start, I will use these questions to figure out the performance tests:

• What is the minimum viable questioning performance tests that you have got to test this feature?
• What is the minimum viable questioning performance tests that you have got to test this workflow?

Unit Tests for Time and Space Complexity

I will work closely with programmers to gather information on below when the code for the feature is committed as part of Unit Tests.

• The execution time taken by the code of that feature - the Big O Notations for space and time complexity
• Usually the Unit Tests focuses on functional tests and clean code practice
• But, when we test team ask and push for performance data, this can come as part of Unit Tests
• An architect or a principal engineer can set an expectation on
• What should be the time and space complexity of a code for a feature?
• Each functions and blocks need to be evaluated on this
• As said earlier, this depends on a engineering practice culture of an organization
• If the culture wants it, it will be there; else just the functional code will be delivered and not the performance code
• If the time and space complexity analysis outcome is not as expected, the code written has to rethought and refactored
• The review process need to put it back
• The comment with data has to be published
• This will be useful to model the performance tests by test engineers who will be working on performance tests
• Doesn't it look a like a effective useful practice as part of Performance Engineering right in the early stage?
• This is very well applicable to projects running on Agile or Waterfall

Do you have this in your project and Unit Tests written?

The time and space complexity questions should not be confined just to the SDETs [test engineer] interview.  A test engineer has to ask for it and apply it in her or his day-to-day work.

Profiling Tests by Test Engineers

We testers do not get into product's code analysis.  We have to build skill to run the profiling on product's code and analyze the resources data.

• Test Engineers can test the feature's code with the help of IDE's profiling (runtime analysis) and collect the performance data by identifying the performance bottlenecks
• This runtime analysis can profile for
• Memory snapshots
• Thread analysis
• Monitoring resources
• CPU and allocation profiling
• And, more
• The problems and risks can be reported upon analysis
• Compare the two different solution's approach performance data

This information will tell and indicate where is the risk and problem when we deploy the code.  In my opinion, this is a useful information in modeling further performance tests.  This information is first-hand information which is very powerful before we start using any other performance testing strategies and tools to aid the tests.

Get Started with Performance Engineering and Tests

These are available in the IDE.  We think of performance testing tools and ask how to test for performance.  To be precise, we test developers (test engineers) should change our mind and shift for first.  If not, as I say, we will be the bottleneck for first to ourselves.  Did you know this way of testing for performance?  Why not you introduce this in your project and organization?

If seen, these test practices can be used right from the day we commit the feature's code. This is a place to start for the performance tests.   This will be a differentiator together with MVPT and guides the MVPT to design effective performance tests in the context.

I do not say these are best practices and there is no best practices.  But this is a useful practice when the organization and stakeholders ask for it.  Let your organization and stakeholders know how well you can test for performance right from first commit of product's code.

To stop and end here,

1. Just do not test for functionality from day one, also test for the performance from the day one.
2. Influence your organization's engineering culture and developers not just for developing functional code, but, also for the performance code

The Agile & Testing: We're Pulled & Hidden from "Shift Right"

Do I Understand the Agile?

If I work in a project which follows the Waterfall approach, I still can apply the Agile Manifesto and the 12 principles behind it.

But, am I adaptable to the change and uncertainty for better ways of developing software and helps others do it?  This shows the mindset of me and others involved.  Have you heard this line, "every project that claims Agile, it has [a small] waterfall in it?".

Agile Software Development is not wrong!  It has values in what it says and for what it stands.  I have no doubts in it!  It describes how I perform a activity and respond to a situation in the software development.

Frameworks and Methodology

We have picked the frameworks which are under the Agile's umbrella. We pick the methodology that suits our work and have customized it to our contexts.

This is what Agile also say, the team will always need to adapt its use of a framework to fit properly in its context.  

Dr. Alistair Cockburn, says, the Agile methodologies are the conventions that a team choses to follow in a way that follows Agile values and principles.

The Agile is Not a Problem!

We do not collaborate well in solving the problem and say Agile is not working and creating more problems.

In reality, how we respond to the uncertainties and what got prioritized, it will derail us.  We derail from what we should be achieving as team [and as an individual] in a given time frame and context.

The Agile as I Understand

In a nutshell, the Agile is about

• Balancing effectively with collaboration in the context we are.
• Identifying, learning and solving the problem as a team no matter where we are in the project's timeline.
• Being ready to face the uncertainty and responding to the changes.
• See the progress and value in what we do as a team and bring the "continuous" in practice.
• Having the consistency in the excellence of our work and the value we deliver with it.

Read the Agile Manifesto and its 12 principles.  It is the day-to-day life of a team who works to build a software in a continuous delivery having the values to a stakeholder.

Testing and the Agile

Testing is a collaborative activity.

The Agile also talks about collaboration as an essence and necessity in the continuous delivery of the software we develop.  

The Agile talks about short interval consistent deliveries.

• How should I fit myself as a Test Engineer here?
• How should I upskill myself to deliver in short intervals as part of continuous delivery?  

I ask and say to my test teams,

• Think of delivering value consistently in the shorter intervals.
• Rather saying -- releases in short intervals, call it out as a value to a stakeholder in regular short intervals as a continuous delivery
• It sounds positive, affirmative and profound.
• Upskill and help your peers to build the skills. List the skills, let us discuss and start our practice on it.
• This mindset changes the perception in how you approach the testing.
• In these short intervals,
• Let us prevent bugs in the ways we can think of.
• Finding bugs also need resources.
• Let's start investing in building and improvising the resources to prevent bugs.
• How can we prevent and where to begin?
• For what should we test and automate other than functionality in these short intervals?
• What is the priority?
• If I do not test and automate for evaluating any particular quality criteria, does it impact the value to a customer in what we deliver?
• If I do not test and automate on a particular seam and system's boundary interface, does it impact the value to a customer in what we deliver?
• How am I collaborating with stakeholders?
• Who are my stakeholders?

In short, I see the Software Test Engineering practice in the Agile setup as this,

• We [team] exploring technically and analytically;
• Evaluating collaboratively in the software development and engineering, by having the effective staged feedback loops in place to improvise the delivery;
• By responding actively to the changes and uncertainties that comes anytime in the timeline of a delivery and practice;
• Thus, the consistent continuous delivery is delivered with value to the stakeholders.

Note: This is my observations so far having worked with 46+ startups in my software test engineering practice.

We're Pulled and Hidden From - Shift Right

I and you are said to Shift Left and start the testing from the inception of a feature's discussion.  And, further we're said, we have to shift left from right.

Did the Agile manifesto and its 12 principles say explicitly to Shift Left from Right?  Who said that?

What I see, How I see and understand the Right and Left here, matters a lot!  If I see the Right within the delivery loop, I'm setting a trap to myself to be late and lag back.  And, probably this trap gets set in the work, that is, to see the Right in the [continuous] delivery loop.  At certain context in continuous delivery loop, starting from Right is a need and necessity.

I see the Right outside the [continuous] delivery loop, as well.  In this Right, I see:

• What is coming up in the tech stack and technology?
• How this is a need?
• How this [is to be and] will be adapted by tech solutions we deliver?
• How should I upskill myself here so that I fit into the loop of continuous delivery?
• In short, the Right should be observing the trend and its analysis so that I'm aligned being agile.
• I will be skilled to test and automate in this change and uncertainties.

The Agile never said to completely move or step out of the Right.  Or, I have not read it in its manifesto and principles. 

You see and learn, what is your Right and ask am I aligning to it for being agile in the Agile.  After all, the Agile says adapting to the changes and uncertainties, yet delivering the value to the stakeholders consistently in short intervals.

If seen closely, the Agile is also about balancing, isn't it?  If I'm moving towards Left and me standing in the Left, am I balanced?

In short, we are being pulled and hidden from Shift Right.

Shifting Right is important to see what is outside the delivery loop.   Are you seeing your Right?  If yes, what is the trend analysis you see?  Where you need to upskill?

Shift Right can also be to look what you can bring into your Shift Left for better development and delivery.  For example, what you see in the market and business space for your product?  Do I have anything to add now in Left from this learning?  Do you see that?  Are not we missing this when we pulled and hidden from Shift Right?

Left or Right, if I do not respond, adapt and deliver for the changes, I need to ask, -- Am I agile and testing in the Agile?

To conclude, know what is your Shift Right while you are delivering values in your Shift Left.  Strike the balance and ace.