There Is No Flaky Test! Then, What Is Flaky?

In general, I understand, the flake means brittle and inconsistent or uneven.  Like the flakes one will have as food.

I'm not a native English.  It took me years to understand and correlate between the words 'flake' and 'flakiness'.

Test and Flakiness

There are no Flaky Tests.  A test is not flaky anytime.

The responses of a system to a test can be flaky.  That is, the response is not deterministic and inconsistent to one's expectation and understanding.  

In other words, the responses of the system is unreliable to the same action, data, state and other parameters.

This is not a philosophy of testing or test.  In my opinion, it is a misunderstanding that is accepted and passed on in the software engineering.

A test is not flaky; the response of a system is flaky.

Are there no flakiness when testing for services?  It exists!  That is, the inconsistent behavior of a service for the same set of data, state and other parameters. What is inconsistent in context of a service's response?

Flakiness is not bounded to just GUI  It can be with any UI [User Interface].  And, an API is an another user interface which opens up to the bunch of services.

To end, 

• It is not Flaky Test.
• It is a flaky response of a system which a test is letting you and me to experience.
• What is a flakiness to one who is testing for security and performance?
• Flakiness is not confined to the UI response or a functional response of a system.

You decide how you should be seeing a test in your suite and label the response of the system you are testing.

Payment Transaction Declined With Code 201, Why?

I read a LinkedIn post yesterday.  This post asks for the appropriate contextual message to deal and how to continue upon what is shown to the user.  I see it is fair and straight.  I expect the same.  

Thanks Liudas Jankauskas for writing this LinkedIn post and sharing your experience.

What surprised me is why the discussion was not taken forward in the post.  As a  result, we did not seed the mindset and attitude of Test Engineering and Prevention discussion.  Testing does not prevent; but, the outcome of the test will persuade the prevention efforts and culture.

Okay! If you see this post is too long to read, then here is in short to you -- This is not a API problem.  The API has worked seamlessly and it has done what it is supposed to do.  The code 201 returned is right and it is supposed to be there.  Here, the 201 is not a HTTP Status Code.  It is 3DS Error Response code.

Then, whose problem it is?  To know that, read the entire blog post.  You will thank me and yourself!

I Say This, Before I Start

1. I'm writing this post as an interpretation and analysis for the JSON shared in the LinkedIn post.
2. There is no intention of pointing to anyone.
1. The intention is to share -- how to analyze and have perspectives to analyze when I [a test engineer] experience it.
2. This post can become a reference to someone who is serious about Test Engineering and Prevention, and Testing.
3. The intention of this post is to let know,
1. How to analyze such incidents in the payment context?
2. Is this a API problem?  What behaviors should be classified as an API problem?
3. Which component and layer is supposed to handle it?  How and why?
4. Which other components and layer is supposed to assist to handle it?
5. How to see and interpret the HTTP and its status code in a narrowed cases?
6. It is not always the API problem!
4. I'm open to correct myself, unlearn, learn and update this blog post, when shared why I'm not making sense with I have written here.
1. I will be thankful and humble to you for helping me to correct myself. 🙏
2. Be comfortable and do connect and help me if you see I need it.

The Payment Failed; Could Not Buy Ferry Ticket

The user wanted to buy the ferry tickets by making the online payment.  The payment did not happen and could not buy the ferry ticket.

Instead, the user sees the JSON on the UI. 

The JSON has error description which reads as -- "TdsServerTransID is not received in Cres."

To understand what is TdsServerTransID and Cres, it is required to understand the payment gateway flows.   Continue reading the next sections to know them.

How It Is Layered and Works?

Before I get into analysis and say where the problem is, this context requires an understanding of the payment gateway flow.  Having this understanding, it helps to know how it could have been handled and prevented.

Refer to this below pic.  It gives the sequences of interactions in the payment transaction.  Note that, the below pic is not complete.  It is kept to what is needed to the context of this blog post.

Transaction between 3DS Server and Bank for the Challenge through
POST request.  It is shown as CReq and CRes.

The Sequence of Interactions

1. I initiate the payment on merchant website or app to buy the ferry ticket.
2. The merchant creates a transaction id for tracking.  Then, it hands over the rest to payment gateway.
3. The payment gateway uses the 3DS way to carry this transaction.
4. The 3DS initiates the Authentication Request (AReq) which will pass through the Directory Server.
• The Directory Server reads the data in the request and forwards it to the right bank who have issued the card [or account] used in the payment.
5. The bank receives the AReq.  
• The bank decides should it give a challenge to the user to make the payment or just agree with the data passed in AReq. 
6. Now, in this case the bank has decided to give the challenge to the user.  
• The bank lets know the 3DS server through the response ARes.
7. The challenge usually is to enter the OTP received over a SMS and authenticate.  I presume the same in this case.
8. The user enters the OTP and POSTS the request.  Let us call this request as CReq.  The CReq is sent from from the user's browser or app to the bank.  The user interface on the browser or app at this point is handled by the payment gateway and not the merchant's web or app where the ticket is being brought.
• This CReq will have,
1. tdsServerTransactionId
2. messageVersion
3. messageType
4. challengeWindowSize
5. ascTransactionId
• Note that, the CReq having the tdsServerTransactionId which is generated by the payment gateway.  tdsServerTransactionId is required in the CReq and expected by the bank.
9. On receiving, the CReq, the bank processes the request.  The bank responds back to the 3DS server; let us call it CRes.
10. The CRes will have,
1. tdsServerTransactionId
2. acsTransactionId
3. challengeCompletionId
4. transStatus
5. messageType
6. messageVersion
11. If all goes as expected, the authorization will be given for the payment.  
• If not, the bank responds to the payment gateway (3DS) that there is a problem and the message.
• It is the payment gateway which has to read this problem and message.
• The payment gateway has to give the better contextual information through its interface to the merchant who is consuming the service.
• This is key!
• Hope you found the spot here!

The Code 201 

Maybe, we test engineers presume the error code 201 returned is HTTP Status Code.  Ask your team what is this code and what is the purpose of returning in the JSON.

Back to 201, here.  It is not a HTTP Status Code.  It is neither a custom developed error code by payment gateway nor merchant who is selling the ferry ticket.

This is a payment domain specific code used in the payment technology.  Yeah!  The code to tell the payment gateway and merchant what happened with the initiated payment transaction.  

To be particular and more specific to the context, the 201 here is a 3DS APIs response code.

The 3DS Error Code and its Description.
The credit of this image is to developer.ravelin.com

The above image is shared here so that you know the error codes are available in the payment context.  I'm aware of 3DS protocol hence I could relate and understand the error code 201.  Refer to the References section at the end of this blog post.

The payment gateway has to read this error code. Then provide the right contextual message and instruction to merchant saying what to do incase of payment failure.

Here is the catch.  Not all merchant develops a mechanism to handle this.  Instead, it is left to payment gateway service provider and depend on it.  

Do the merchant bother about handling it at their end? 

• As a merchant, I can switch to another payment gateway tomorrow.  
• Why should I invest in building, developing and maintaining this as a part of my business when I'm paying another business to do it for me?  
• Isn't that senseful business question or decision?

Now, you tell me who is expected to handle this 201 error? Who should let me know on the merchant web user interface what to do?

The Case and My Initial Observations

On submitting the challenge given by the bank, the below details is shown to the user.  

Pic used from the post of Liudas.
It is the CRes returned from Bank to 3DS Server at Payment Gateway.

My Observations

1. Looks like the ticket being bought and payment made was in Turkey.
2. The payment gateway involved to complete the payment transaction is Payten.
3. The user is tryin to buy the ferry tickets and is making online payment. This error is seen on submitting the challenge given by the bank,
4. Looks like the card is used for the payment.
5. In the LinkedIn post, I read, multiple attempts were made to make payment, and the same message is seen.
6. The Error Code returned is 201.
• Which means, a mandatory field is missing and it is a violation.
7. Error Component is S, which probably means 3DS server.
• The component that raised the error.
8. There is an error which is not handled.
• The error reads as Unexpected error.
• There is an id to track it.  That is, for auditing and tracking by the payment gateway.
• This id do not have anything to do with the bank and merchant who is selling the ferry tickets.
9. The message has a version that reads as 2.1.0
• This means, 3DS protocol used is of version 2.1.0
• This is communicated by 3DS server to the Directory Server and Bank Server, so that they use the same version in the contract.
• If the same version is not available at the bank server side, the lower version will be used.
10. Error description reads, the TdsServerTransID is not received in the CRes.
• The 3DS component is raising this error.
• TdsServerTransID means ThreeDS Server Transaction Id
• 3DS means Three-Domain Secure, which is a security protocol used for online card transactions.
11. This means, the response from bank is, it cannot fulfil the payment request.
• Because, the request from the 3DS layer do not have TdsServerTransID.
• Wait!
• Do I actually know if the CReq had the TdsServerTransID in the payload?
• This is critical to know.
• This is important to know.
• The AReq (Authentication Request) will have the TdsServerTransID in its payload.
• That means, this was present earlier.  So, the CReq is fired from the 3DS server on receiving the ARes.
• The bank received TdsServerTransID and said it is giving a challenge  before proceeding to authorization for payment.
• This response (ARes) from the bank will have TdsServerTransID.
• Along with this, the below are also present as a correlation ID to track this transaction.
• dsTransID
• The Directory Server ID to track this transaction
• acsTransID
• The bank server ID to track this transaction
• From here, the CReq payload requires the TdsServerTransID and acsTransID
• Where did the TdsServerTransID go missing now?  How?  Why?
• This needs to be test investigated.  I have shared possible causes in the next section.

The TransactionID of the merchant who is selling the ferry ticket is different from the TdsServerTransID.

• The TdsServerTransID is needed for the bank and Payment Gateway to complete the transaction.
• Without this TdsServerTransId, the bank cannot proceed to authorize the payment.

All this is happening between the payment gateway [3DS server] and the Bank.  The user and the merchant website is not in the picture here.

Why Did It Happen?

It happened because, the CReq from the 3DS Server of payment gateway did not have that TdsServerTransID.  That is what the CRes from the bank says.  

Note that, I presume, the bank systems and services are functioning and serving its customer at this point in time.  This is an analyzed assumption I'm making here and I'm aware of this assumption I have made.

What made the CReq to miss the TdsServerTransID?

1. I read, the user made multiple attempts to initiate and make the payment.
2. I have multiple perspectives and interpretations with hypotheses to say this could have lead to miss the TdsServerTransID.
• Talking about each hypotheses in detail is not the scope of this blog post.
• But, here are some of the possibilities that could have led to this situation.
• A glitch at payment gateway at that point in time.
• Someone is breaching in the middle and tweaking the request and response over the network.
• Device and hardware
• Geo location
• Network and traffic
• Timeouts
• Latency
• Storage running out
• Configuration gone wrong
• Caching and missing -- not persisting
• Intermediate and dependency services clogged
• A new release deployed and updated
• Downtime in the bank's services
• Intermittent bank's services
• Mismatching 3DS protocol versions that is not supported and accepted at either ends
• And, more!

In simple, it was well handled by the bank.  I assume, the amount is not debited from the user's account.  This is equally important and the LinkedIn post do not share about it.

I see the bank's service have worked well in this context and done what is expected out of it.  The thumb rule is, when there is discrepancy in the data expected and received, do not authorize the payment request; abort it. 

Can the service do much more if the TdsServerTransID is missed in the payload of AReq?

Me as a test engineer in the payment gateway engineering team, I will test for the below in minimal and as a must,

1. AReq with no TdsServerTransID and observe how what happens!
2. Do the 3DS server still fire a AReq to the bank with no TdsServerTransID?
1. If yes, then that is a problem!  It should be handled here.
2. This problem can be prevented!
3. I will ask my team why are we encouraging such request?
• This increases our customer support cost and operations time in responding to our clients for using our payment gateway.
• And, merchant can lose the business because of our payment gateway.
3. What should 3DS server do when initiated AReq is missing the TdsServerTransID?
1. What all other data in the transaction and session should be retrained and intact?
2. Will these data change with a fresh TdsServerTransID created?
4. I will explore and figure out what factors caused the 3DS to lose the user's authentication.

Who Created The Problem?

1. From the error shared, I see,
•  It is 3DS server who created this problem presuming the bank and its response is right.  

Is This An API Problem?

1. No, it is not a API problem.
2. If we call it as a API problem, we have not understood what is an API.

Whose Problem It Is?

1. It is the problem of the service that fired the CReq.
• Because, it fired CReq without the TdsServerTransID which is mandatory key-value in the payload.  
• TdsServerTransID value cannot be null nor empty in JSON.
  

It appears to be a business logic problem of the service at 3DS server side for firing CReq with no TdsServerTransID.

How It Can Be Prevented?

1. By making sure CReq will always have a distinct TdsServerTransID.
2. If no distinct TdsSevrerTransID in a fresh CReq being fired, abort the request.
• Create the distinct TdsServerTransID and then construct the request payload before firing the CReq.

To make sure, payment gateway [3DS server] preserves the TdsServerTransID, dsTransID, and acsTransID of the session.  If any one of this not matching at any of the components during the transaction, aborting the transaction is the best and right action to do.  

Can you recall your daily life experience where you are said to not refresh or click on the browser's back button when you have initiated the payment transaction on web or mobile app?  This is the reason!  To preserve the data of the transactions in the session between these systems -- merchant web or app, payment gateway [3DS server], Directory Server and Bank Server.

This is an automation candidate.  It has to be part of the daily test runs in the automation.

Who Should Be Fixing It?

1. In my opinion for today upon analysis and assumption I have made, it should be fixed by the payment gateway.
2. The payment gateway has to interpret the 3DS error code returned by the bank.  
• And, then initiate the appropriate action as the payment transaction has failed. 
3. A new transaction between the 3DS and bank has to be started for that merchant's order.  
• If it cannot happen, the payment gateway should abort all the current open session tied to that merchant's order.
• And, let know the user what is happening, and then direct the user.
4. The payment gateway can read the ARes and CRes.
• That means, the payment gateway can read the HTTP Status Code of ARes and CRes.
• If there is an error code in ARes and CRes,
• Then, the gateway should assert for the 3DS error code along with the HTTP Status Code.
• For example, 
• Say, the HTTP Status Code is 400 and 3DS Error Code 201 in CRes.
• Assert for these two and direct with appropriate contextual message and direction.
• In general, this is how custom developed error codes are handled by the client on receiving it from the services.

Note that, it is Error Code and not the Response Code.  The two are different.  And, the HTTP Status code is different from these two.

Why It Is Not An API Problem?

API is an interface which exposes the available services to the consumer.

It is the services which collects data and build the request payload, and expects the payload to process. 

This request will pass through an interface which is opened to the consumer.  This interface is called an API -- Application Programming Interface.

Analogy,

• The car has gear stick to switch the gear.
• This is an interface to the car's gear system.
• The driver will use the car's gear system through this gear stick.
• The gear box in the car is a service.
• The gear box adjusts and responds by switching to the gear per the driver's input.
• The different operations [business logic] provided by the gear box services are,
• Reverse
• Parking
• Neutral
• Switching the gear and assisting other components to speed up or speed down the car.

Can the interface have problem?  Yes, it can have.  In that case, the service will not be available to serve or to discover.  

• Like, if the gear stick has problem, I cannot use the gears of the car, but, the gear box can be fully functional and in working condition.  
• It is just the interface [gear stick] having the problem and as a result the consumer [driver] is unable to use the services [gear box].

In this case, the payment gateway could fire CReq to the bank.  That is, the API exposing this service is functioning.  It is the problem in a service.  It is the service that has missed to ensure and mandate the presence of TdsServerTransID -- a business logic problem.

In simple, we Software Engineers [including me] use the term API vaguely and with no sense of what it means.  This is my observation.  Further, the Software Testers have tossed this term API in all possible ways and learning it in incorrect ways.  I have no doubt in it when I say this.

Next time, when you say it is a API problem, rethink on what you are saying.  Is it an API or the service(s) which is accessed through that API?  It is useful when we describing the behavior of the system and its layer.  

I cannot say it is a gear box problem for the gear stick not [usable] working, and vice versa.  You see that?

To stop here,

1. Testing skills and testing will help when it is collaborated with awareness and skills of the tech stack used in building the software system.
2. Testing skills, programming and tech skills are not enough!
• The domain skills and awareness is essential and critical.
• If one is aware of the domain where the problem is observed, then the 201 will not be read as HTTP status code.
3. Build the domain skills and maintaining the knowledge base as a GitHub project helps in a longer run.
4. Interfaces and its gateways should not be overloaded with additional responsibilities to make sure the mandatory key-value is present.
• If did so, one should learn why. Because, it is an anti-pattern and not a suggested software engineering practice.
5. It is the services that has problems most times.  
• The interfaces and its gateway will have the discovery, orchestration and traffic problems along with the risks of security.

References:

1. https://httpstatuses.io/
2. https://developer.elavon.com/products/3dsecure2/v1/api-reference
3. https://developer.ravelin.com/psp/api/endpoints/3d-secure/errors/
4. https://developer.elavon.com/products/3dsecure2/v1/3ds-error-codes
5. threeDSServerTransID  (in our case the JSON reads as TdsServerTransID)
• 3DS Server Transaction ID
• Universally unique transaction identifier assigned by the 3DS server to identify a single transaction.
• The 3DS server auto-populates and appends this filed value in the authentication request (AReq) it sends to bank in addition to the data you send.
• This value can also be found by a lookup in the response received by this service -- /3ds2/lookup

The { } and Lessons I Observed

 

I did not know { } are called braces.  I called them curly bracket or flower bracket in my verbal communication until I started my career.

In the early days of my engineering college, I wondered why the braces are used and not the square bracket or parenthesis to open and close the class, function and a block of code.  

I was practicing C, C++ and COBOL then.  Though COBOL did not use braces, it used indent to start from specific column in a row.   But, the philosophy of a programming language construct looked relevant in COBOL as in C and C++. 

When I practiced Core Java, I could relate the programming constructs and find the similarities in C, C++, COBOL and Java.  So with C#, JS, and Shell Scripting.

All this said, the question that remained in me is -- Why these programming languages use braces to open and close the class, function, loop & conditional blocks?  Why this similarity?

When I studied Discrete Mathematical Structures in a semester, I could correlate the programming to it.  This helped me to reason and interpret why the braces in a programming language.   

Pic: Programming languages having braces to open a function.

{ } and Set

In the Discrete Mathematical Structures, I studied Sets. Further, I learned how the Sets is used to classify and organize the elements in problem solving.

While I was solving a discrete math exercise,  this struck to me -- Why the braces to represent a SET?  This triggered my associative memory and sparked a question -- Why do I see the braces in C, C++, and C#?

Here is how I interpreted and reasoned my thoughts and questions.

• The SET is to represent the elements of common nature and characteristic which has a purpose. Most of the times it is deterministic.
• But, why the different programming languages use this braces to group a set of instructions? Why not something else to represent programming instructions with a common and deterministic purpose?  
• The COBOL do not use braces, but, the philosophy and thought process of braces exist in it as starting from specific column.
• I observed Java and Shell scripting using the braces.

The point and thought to note here is, the two words -- SETS and Groups. I'm studying this since my primary school days.  But, I had not realized how they are so much into the computer science!

Today, I observe most programming languages use the braces to group the set of instructions which are for a specific purpose.  Be it a class, function, loop or a conditional block, they are within { }.  

Between the { }

The instructions written between the braces has an objective.  Be it class, function, condition or loop block.  That is, I group this instruction as a set.

Why I call it as a set?

• The instructions between the braces have the one purpose.
• Is this SRP -- Single Responsibility Principle?  I am yet to put this in a better word!

Is the set a collection of elements that has a common characteristic and purpose?  Then, between the braces, the code has a common [one deterministic] purpose.  

Does this thought of programming language creators made to use the braces?  If so, this is a beautiful learning which is missed to observe and interpret.

{ } and Programming Languages

A programming language is created [written] to overcome the limitations of other language in a context.

Yet, each person who created the respective languages did agree to a common representation and understanding of semantics and constructs like the braces.  I see this is missing in Software Testing fraternity.  

Though, the creators of programming languages have a differing thoughts on other programming languages and what it takes, they have something in common to represent their thoughts.  

Looks like the most creators seem to go with this common representation.  The braces is one such example.  

Their vocabulary is something amazes me; most don't seem to have a problem with it.  The idea and thought are the same behind the vocabulary in most cases.

Another example is how to write a comment.  The programming and scripting languages that I have used so far uses // to comment a line.

To end this post here, personally I feel, I as a Software Test Engineer have a lot to learn from the programming fraternity.  

The next blog post is continuation of the thought shared here and sharing my experience of how the software testing fraternity looks to me in comparison to programming fraternity.

Testing and Programming Are Not Separate Disciplines

 

Do you and your team see Programming and Software Testing as two separate disciplines?  You believe that programming and testing are as different as oil and water?   If so, this blog post will converse with you to realign your thoughts.

When I started to draw the correlation between Software Testing and Programming,

• That is when I realized Programming is beyond writing the code.
• Testing is an integral part of programming.
• Writing code is one part of programming.  
• Likewise, testing the written code and the associated system, is part of programming.  
• I see, the idea of TDD is an example to it.
• Testing is not the other side of a coin.
• Programming and Testing are on the same side of a coin!  Testing helped me to learn this.

Then, what is on the other side of the coin? 

• It is we on the other side of the coin, 
• Who are seeing and advocating testing and programming are the two different disciplines.

Had I understood this early in my career, it would have made a significant difference.

For the last 10 years, I have been assisting myself and mentees to experience this perspective of programming and testing.

The sad part is, the colleges and universities do not tell this to their students.  Instead, we are said the programming and testing are two different stuffs in software system and engineering.  Thereby, treating the one is superior to other.  Eventually, we in the industry continue to share the same thought and dialogues in all spaces.  Thereby, creating the gap in skills and thoughts of both programmer and tester.

Programming 

What is Programming?

A program is planned and an ordered event of actions with a purpose.  When it is ordered in a specific way, it involves logical decision of a programmer to design and implement a program in a intended way.  

In Computer Science, I accomplish this with the help of programming languages and commands which an operating system leverages and executes.

Programming is a thought process and thinking activity which evolves over the time with discipline and consistent practice.

• It involves thinking, understanding, questioning, and analyzing the intent of learning and solving a problem in hand.
• It demands questioning and learning the context.
• It involves studying of -- systems, people, environment, communication, programming languages, design, modeling, simulation, engineering, technologies, anthropology, geography, economy, business, politics, laws, regulations, computer software, software tech stacks, hardware, domains, different professions, jobs, emotions, crime, psychology, science, arts, and you can add more to this list as you unlearn and learn.
• It demands analytical thinking, critical thinking, and ability to develop clarity for making decisions to design the implementation and code.

Programming is like a manifest file in Android app, where it manifests and expects how and what the app should be like.   The app code implements the manifestation.

Coding

Visualizing and writing the programming instructions from the manifestation thought of programming is coding.  The programming instructions [logical] are written using programming languages.  I understand it, this way.

To code, one uses programming languages.  Further one has to understand the Data Structures and Computer Hardware and associated software to code efficiently and effectively.  This is where I understood the need and value of the Data Structures.  This forms the basic and fundamental necessity for one who wants to build and test the software system.

Each programming language has its own,

• Syntax
• Rules
• Limitations
• Way of execution
• Space
• Time
• Storing and retrieval of data

The below said will be well evaluated for the context,

• Which programming language to code? Why?
• What Data Structures to use for processing the data?

That said, the idea of debugging remains relatable and same in most cases irrespective of the programming language.  

That is, the debugging is at meta layer of programming.  Based on the context, how the debugging is done, can change with the programming language.  But, the idea of debugging remains the same.

Programmers learn and practices different programming languages.

Programmers solve the problem using programming language.  To solve, programmers start visualizing with programming, and then code.

I see, the programmer learn -- Is the actual problem being solved?  So that the time and efforts are well spent.

To stop here, there is,

• Clean Code, and not Clean Programming
• Code Smell, and not Programming Smell
• Code Refactoring, and not Program Refactoring

Programming and Coding are two different skills and it runs in parallel helping each other.

It is like, the programming is a blue print, and, coding is the construction work.  Both are needed!

Testing

Having one's own definition as a practitioner is good.  But, then I see, my definition also aligns with that of Dr. Cem Kaner, James Bach and Ashok Thiruvengadam.  So, I quote their thoughts here.

Testing is,

• Questioning a product to evaluate it.
• Probing the system to learn and evaluate it.
• Empirical technical Investigation
• To learn quality related information of a software system
• So that the stakeholders can make informed decision

I see, testing is an another stand to the programming.  It is not different.  

Programming and Testing are analyzing and thinking skills for first.  The thinking applied and implemented and evaluated is coding skill.

What all needed for better programming, all those are needed for value adding testing.  Refer to the above section on Programming.

Further, testing uses coding to execute the intent of a test to some extent.  And, this is a need.  Being skilled in programming and coding gives a different perspective and strength to the testing.  The value out of this will be distinct and prominent in most context.

If one can test, one can program too.  

Now, do you see that the programming and testing are not the two disciplines?  They are one!  They are on the same side of the coin.  On the other side of a coin, it is we who are not willing to see this and advocating these two are different.  

I learn, a skilled tester can add keep adding ways to sample, interpret and test when she or he can use the coding skills.

To summarize,

1. Programming and Testing are the same discipline.
• Seeing it as two different discipline is counterproductive!
• This counterproductive can be seen in the software industry, today.
2. It is useful thought process, mindset and attitude to see both programming and testing as a overlapping areas and one discipline.
• Both are on the same side of the coin in Computer Science, Software & Engineering
3. Coding skill does not mean one has programming skill.
4. Coding skill does not mean one has testing skill.
5. Programming skill shows the testing skill and vice versa.
6. Programming is a way of testing.
7. Programming is a way of debugging.
8. Testing is a way of debugging and refactoring the programming.
9. Programmer develops the tests and code.
10. If you can program, you can test.  If you can test, you can program.  If you can program, you can code.
• But, coding skill alone is not enough to program and test.
• These all are three distinct skills, where programming and testing overlaps very much on each other.

Empathy - Missing in Engineers. Then, Why Think Like a User?

I did not know about the word 'empathy' in English.  I heard it for first time 15 years back.  This does not mean, I did not have or express or share the empathy.  

We engineers talk about empathy to the users when building the software.  That is, think like a user.  If you are a test engineer, testing, then you should have heard this.  And, you should have asked yourself -- "How will the user feel with this?".  Did you?

When you write a bug report or any test report, you think and ask yourself, "Can this be easily understood by the reader?".  Won't you?

You are having the thought and an emotion of empathy in that thought and question!

How's the Empathy of Engineers for Engineers?

Anytime, you asked these questions to you?  What do your voice say?

1. Do I truly listen to my colleague and peers?
2. Do I document?
• How well and lean I document so that fellow engineers can use it?
• Can they use it in my absence?  Does it serve?
• For example, 
• How well I name the variables, so that, someone who is reading my code can make sense out of it?  This is empathy to your colleagues.
• How well I write the test engineering related testware, so that, it serves my fellow test engineers and others?  This is empathy to your fellow testers.
• Before thinking like a user, think -- How my fellow testers feel for using your work and communication?
3. Do I consider how my words and behavior affect the people I work and collaborate with?
4. Do I listen and respect my colleagues and peers view while I discuss and share my perspective when it differs?
• Am I ready to accept the same treatment and communication that I share with others?
• Do I share and communicate my perspective and intent for first?
5. How do I interpret other person's understanding and thought about a subject, work and practice?

If you are a programmer, 

Ask yourself when and how you respected your fellow programmer and testers thoughts and work?

If you are a tester, 

Ask yourself when and how you respected and listened to your fellow testers word and work?

If you are a manager or director or VP or CxO role person for engineers, 

Ask yourself, when was the last time you listened and tried to understand the words and thoughts of your programmers, testers and other roles?  How did you listen and communicate?

• How did you help your teams and a team member to understand your point?
• Did you consider one of your team members as your team?
• Did you say anytime, she/he knows better than you, to any of your team member?
• Did this end or impact the communication thereon between you two?
• Further, ask yourself and list out the incidents that you could have avoided a situation and communicated well with dignity?

It is easy to say the slogan, "Think like a user".  

It is much easier to say this to your testing team.

But, it is not easy to think from the point of your colleague or peer.

Why?

What stops you?

Empathy Starts Within - A Message to Engineering Leaders

As leaders, we often say,

"Think like the user."

It is a powerful principle.

But, here is the question.

"Do you think like your engineers?"

You hired your engineers by conversing with them.  What are your efforts to make your engineers understand your thinking?  What's your principle on this?

If you cannot think and respond to the point views of your colleague or peer or team, how can you think and build the software system for a user?  Yet, say from decades, "Think like a user"!

Let me ask this.  Anytime, you tried to impose or overrule the testing team's thoughts, pain, voice and perspectives?  Or, did you talk to them to understand it and then shared why it should be the way you are deciding?  

Did you think as your test engineers and test team?  And, you want them to think like a user?  How?  

I'm not saying to agree and acknowledge to whatever the test team say and ask.  But, are you aware that, you work with test teams?  They are one among who consumes your decisions and work to deliver as a team.   What's your empathy to them?  Also reflect on what's your empathy to other teams?

If you cannot communicate well and help them understand your points and need of the business, then, how effectively as a team can we deliver value to the users who buys the service of the business?

When no empathy to fellow engineers on the team by an engineer, and, by the one who is heading the engineering teams, how can one have and express the empathy for the users who buys the service of the business?

Do you have the empathy for the engineers in your team?

Do you have the empathy for the teams you lead?

What is empathy?  What is your empathy?

I understand, in the business and with an executive role in business, empathy has scale.  What's your empathy scale for your engineers?

To end here for now, before the users, let your engineers and teams experience your empathy.

 

Is Software Testing a Cost to Business?

How quick one gets to have the awareness of the cost and value in the trade, it will be of help.  I consistently exercise the below questions in my practice.

• What are the Values added from my work to the business?
• How do I benefit from this?
• What are the Values removed from my work to the business?
• What does it bring to me?
• Who is not getting benefited from the Values I'm adding and why?
• What are the loses and its impact?
• What are the benefits and gains we are losing?
• Who is getting benefitted from the Values I'm adding and why?
• What are the benefits and gains we are making?
• What are the Costs added from my work to the business?
• What are the impacts?
• How does it impact me?
• How does it benefit me?
• What are the Costs removed from my work to the business?
• How does it benefit me and what do I gain?
• How does it benefit the business and others?  What do they gain?
• Who are all experiencing the Costs from my work and why?
• What are the pains from these costs?
• Who are not having any Costs from my work and why?
• Do they need anything from my work and its value add?

This exercise will help me to know the expectations of stakeholders and business. I align myself to deliver the expectations as I exercise these questions.  

The cost and value from my software test engineering work will have impact and influences my growth with the benefits that I will make.

Is Software Testing a Cost to Business?

I witness the discussion which says testing is a cost in software engineering business.  But, I do not hear the same statement on development.  This made me to think, why?

Here are my understanding on thinking over it for now.

1. Software development is not about just programming.
1. When said development it includes every teams and their work.
2. Programming and Testing are parallel activities in the software engineering which helps each other.
3. If testing is a cost, then for sure, programming is also a cost!  It is an associated activity.
2. In business, every activity and investment on them is a cost in multiple ways.
1. These are evaluated costs and being taken for a purpose in expectation of returns.
3. Have I come across anyone saying Automation in Testing is a cost, like I hear for testing?
1. It is seen as investment and necessity. Why?
1. May be, because, in a belief if [once] automated, that is sufficient enough it goes on for itself without human intervention.
2. If this is the thought, do you think we are doing it wrong?
4. Have I seen the discussion and statements which concludes testing for performance and security is cost?
1. If yes, how?
2. If no, why these two are not seen as cost?
5. Serving and well maintained engineering system is a trade-off
1. I see, we engineers and business choose what to trade-off
1. This decision can be on logical and non-logical basis; but, there will be a trade-off
2. Each trade-off has costs and values
3. What should I trade-off in the software test engineering and why?
4. What should I trade-off in the software engineering that we are doing as a business?  Why?
6. When one is offered a offer letter from a business, there is a CTC mention.
1. CTC means Cost to Business
1. Programmer has a CTC
2. Tester has a CTC
3. Every others in different teams have a CTC including the CxOs
4. We all [and our job] are cost to a business, who can add value and get the high returns

To end for now, I learn, everything and anything business picks up is a cost.  Because, there is an investment on it.  If there is no investment, is there anything happening or progressing and changing?

I do not want to get into discussions which says software testing is a cost.  I see such discussion is lacking in understanding the software engineering and intrinsic for first.  Software Development is not all about programming; the programming is one small part of it.

There are different teams that work together in collaboration to develop and consistently delivering the usable software systems.  In this process, everything in software engineering business is an invested and evaluated cost in the interest of returns and values that are expected.

It is time to know and ask "What way of testing and its outcome is a cost?" than saying and listening to -- 'testing is a cost', which is nonsensical.  This holds good for any activity in software engineering.  

Any serving engineering marvel is a calculated and evaluated trade-off.

Backtracking of Testing, Security and Tools

 

When I started my software testing career in 2006, I was in this thought -- What tools should I use, so that,

• I can do the testing that is sought after
• I can test for performance
• I can test for security

Moving from a search for tools to building the mindset and attitude.  It is a journey!  It took me time to see this journey.  I hopped on to this journey in 2011.  I see, this is not an ending journey, while I know where should I go and reach.  I'm on this journey.

I had no mentors.  I had no seniors in software testing to guide and discuss on my thought process.  I had developers (programmers) who had little or no interest in testing; so it did not matter to them.  But, they have helped me to be better tester.  I'm grateful to them.  Then, the community was not so connected, organized and share the knowledge as it does in 2024.  The software testing was not considered or seen as a technical activity, then.  I have stood, fought, demonstrated and delivered my testing as a technical activity.  I'm continuing it.

Today, on 24th Feb 2024, I read the below question in a community's social space and decided to write this blog post.

Hey, everyone .... Can anyone please suggest a good tool for API security testing?

This question resonates in test engineers.  Most of we test engineers still look and ask for tools when it comes to security testing.  To test engineers, the performance and security testing are still a conception and activity with tools alone.  In reality, it is not!  If you are in such thought or you come across such question to answer, this blog post is for you.

Backtracking the Problem Identification

In programming, we have an approach by name Backtracking.  It is about exploring in possible ways to find possible solutions for a problem.  And, a best solution which works in context is picked.

What's the problem here?  Testing, Security and Tools. Are you with me so far? Let us backtrack this problem.

Note: I see a difference between the words 'possible' and 'all'.  Hence, I use the words "possible ways" and "possible solutions" and not "all ways and all solutions".

Bounties and Entry

There are reputed bug bounties for security testing.  To get into this bounties one has to showcase her/his discoveries and skills with her/his recognized portfolio.

The tools are accessible to all.  The community edition and licensed edition tools are available.  We use these both editions of tools.

• But, why not all of us with tools cannot get into such invited security bug bounties?  
• You will answer this question if you ask yourself.  Hope this backtracking should have helped by now!

The Security Engineering is a vast practice area in Software Engineering. There are dedicated security engineers in role.  But, we test engineers can take up the testing for the security of software systems which the team is programming and building.

I advise, a practicing test engineer

• To start with building an interest for security engineering.
• Consistently hone and build the mindset, attitude and skills needed for the testing the security aspects.
• Pick simple problems, solve it.  Do it consistently, while you explore the layers.

While this is done consistently, it is time to find the mentors in Security Testing. The mentors will assist you in practicing how to test effectively for security making use of simple contextual necessary tools.  Also, a mentor will let you know how to test for security without tools to an extent.  The tool is effective when known how to use it.  The tools help immensely only if I can test for security. 

To backtrack in a different perspective, did any tool that you use, find a P1 security problem [or risk] by itself in its scan?  Did your programmers acknowledge to that risk or problem?  I will pause with these two question to you.

Today, my testing for security is confined to systems that I test.  I test for web application, mobile apps, web APIs, and database.  I can assist here, if you do the home work and ping me.

Performance Testing - The Unusual Ignorance in Practice & Culture

 

I'm continuing to share my experiences and learning for100 Days of Skilled Testing series.  I want to keep it short and as a mini blog posts.  If you see, the detailed insights and conversations needed, let us get in touch.

The ninth question from season two of  100 Days of Skilled Testing is

What are some common mistakes you see people making while doing performance testing?  How do they avoid it?

Mistakes or Ignorance?

It is mistake when I do an action though I'm aware that it is not right in the context.

I do not want to label what I share in this blog post as mistake.  But, I call it as ignorance despite having or not having the awareness, and the experience.

The ignorance said here is not just tied to the SDLC.  It is also tied to the organization's practice and culture that can create problems.

To this blog post's context, I categorize the ignorance in these categories -- Practitioner and Organization.

1. Practitioner's ignorance
• Not understanding the performance, performance engineering, and performance testing
• When said performance testing, taking it as - "It is load testing"
• No awareness on what is performance and performance engineering
• Going to the tools immediately to solve the problem while not knowing what is the performance problem statement
• Be it web, API, mobile or anything,
• Going to one tool or tools and running tests
• No much thinking on how to design the tests in the performance testing being done
• Ignoring Math and Statistics, and its importance in Performance analysis
• No idea on the system's architecture, and how it works
• Why it is the way it is?
• The idea of end-to-end is extended and used in testing for performance and having hard time to understand and interpret the performance data
• How many end-to-end your tests have identified?
• Can we test for performance to all these identified and unidentified end-to-end?
• Relying on the resource/content in internet and applying or using it in one's context without understanding it
• No idea on the tech stack and how to utilize the testability offered by it in evaluating the performance
• Not using or asking for testability
• Getting hung to most spoken and discussed 2 or 3 tools on the internet
• Applying tools and calling out it as performance testing
• No attempting to understand the infrastructure and resources
• How it impacts and influences the performance evaluation and its data
• Idea on Saturation of resources
• Thinking it as a problem
• Thinking it as not a problem
• Not working to identify where will be the next bottleneck when solving a current bottleneck
• What to measure?
• How to measure?
• When to measure?
• What to look when measuring?
• Not understanding the OS, Hardware resources, Tech Stacks, Libraries, Frameworks, Programming Language, CPU & Cores, Network, Orchestration, and more
• Not knowing the tool and what it offers
• I learn the tool everyday; today, it is not the same to me compared to yesterday
• I discover something new that I was not aware of what it offered and exist
• I learn the new ways of using the tool in different approaches
• No story in the report with information/image that is self-describable to most who reads it
• And, more; but the above said resonates with most of us
2. Organization's ignorance
• At the org level, for first and to start, it is ignorance in Performance Engineering
• Ignoring the practice of performance engineering in what is built and deployed
• Thinking and advocating, increasing the hardware resources will increase and better the performance
• In fact, it will deteriorate over a period of time no matter how much the resources are scaled up and added
• Ignoring the performance evaluation and its presence in CI-CD pipeline
• The performance tests on CI-CD pipeline should not take beyond few minutes
• What is that "few minutes"?
• Not prioritizing the importance of having the requirements for Performance Engineering

Recently, I was asked a question - How to evaluate the login performance of a mobile app using a tool "x"?

In another case, I see, a controller having all HTTP requests made when using web browser. Running these requests and trying to learn the numbers using a tool.

I do not say this is wrong way of doing.  That is a start.

But, we should NOT stay here thinking this is a performance engineering and that is how to run tests for learning a performance aspect[s].

To end, the performance is not just - how [why, when, what, where] fast or slow?  If that is your definition, you are not wrong!  That is a start and good for start; but, do not stick on to it alone and call performance.   It is capability.  It is about getting what I want in the way I have been promised and I expect; this is contextual, subjective and relative.  The capability leads to an experience.  What is that experience experienced?

Sometimes, serving the requests by what you call as slow, is a performance.  What is slow, here?

The words fast and slow are subjective, contextual and relative.  It is one small part of performance engineering.

That said, let me know, what have you been ignoring and unaware in practice of Performance Engineering & Testing?

A Test Is Not a Metric

 

A test execution by human or automation will provide information to be aware & learn.

If there is a metric, it should be for what I got to be aware of & learned.  Not for the number of tests.  If it is for a test, it is blunder, before taking it as a metric.

One can identify infinite tests; automate adding annotation @test in big numbers. Should this number be a metric?

That is a question to ask when you see a metric on this number.

What is of value to you in this outcome from a test?  That value should have a metric, and not test or the number of tests.

Identifying the metric in a context which serves, is not easy.

Number of pass/fail or green/red is a measurement; not a metric!

Anything measured cannot be a metric.

A metric helps to measure in a way that establishes rational correlation & upholds its necessity in business.

Share this awareness!

Behind the Every Test Data, There is a ?!

 

Read this blog post to have a perspective about the Test Data and Test Data Management.  The point is, if I'm not aware of a test and what does it tell me to explore, I cannot think of a Test Data.

That said, if I know what I should be evaluating as part of performance, why, when and how, this will help me to come up with a thought for identifying the tests and its test data for the same. 

The ninth question from season two of 100 Days of Skilled Testing is:

What role does data management play in performance testing, and how do you ensure the availability of suitable test data sets?

Testing and "Ensure"

We test and have tests in testing, because, there is no "sure" and "ensure" idea in software.  But, we presume on a rational basis upon, "if, these are this", in a given context when the software processes.

Now, ask yourself, how can we ensure the availability of suitable test data sets?

In my opinion, the Test Data is often misunderstood.  This is the primary problem and should be the first problem, when asked "what are the challenges in creating the test data?".

When you read the concluding lines of this blog post, you will learn why I say this.

Test Data and Immunity

In my opinion and experience in practicing the Test Engineering, I see, the Test Data should be a viral strain and it should have its variants.  When this test data is used to test [experiment, test investigate, and debug], how do the software and its ecosystem respond?

• Does the software and its ecosystem is immune to this test data?
• Does it exhibit any risks and problems?
• If yes, then, do the purpose of my testing and automation is accomplished with this test data?

This puts me back to question, what is the purpose [intent] of my test?  It drives me to derive the test data which helps me to know -- What am I supposed to learn and on priority?  With this, I get an idea for what kind of test data I should be creating knowing its pattern.

If the system is immune to Test Data and not reveling anything new in the context, I classify this pattern of test data as "Immune" to the context.

In my practice and research work in Test Engineering and Software Testing, to start, I categorize Test Data into two areas.

1. Immune
2. Not Immune

Further, I have categories, under these two, where I classify the Test Data deterministically for the context.   Get in touch if you want to learn more about this.  I'm just one ping away!

The tests should help me to evaluate for the immunity and also non-immunity; both are essential and necessity.  

The credit is to me for such classification of Test Data.  It is my research work out of my practice.

Note that, Test Data is not just the input [characters or files] entered or given to a system.  Test Data has its association to tech stacks, infrastructure, ecosystem, business workflows and people.  To craft such Test Data, one has to have the understanding of the system and its internals, and, the problem it solves by knowing how it solves.

Performance Testing and Test Data

1. What is that I'm testing as part of performance?
2. What do I want to evaluate in the name of performance?
3. What part of the system is evaluated for its performance?
• Should I evaluate this in isolation or as a wholeness of the system?
4. What domain knowledge and information I should have when testing for performance?
5. What system's architecture and internal details I should understand and be aware to test for performance?
6. Is this the first delivery?  Or, do we have this system running in the production?
• If it is first delivery,
• How will I create the test data to suit the consumers of this application?
• What are the key workflows of business that we should be evaluating for its performance?
• Do all workflows and sub-systems need the evaluation for performance, and on priority?
• How do I map the fragmentation of users and their data [with its patterns]?
• What are the infrastructure and ecosystem characteristics that should be part of the test data identified?
• Does caching have any effect if the same pattern of data is used?
• If it is a running version in production
• Can I refer to the DB to figure out the pattern for the particular workflow that I'm evaluating?
• How can I match the test data to have the production data's characteristics and attributes?
7. What is the backup strategy for the Test Data?
• How do I version control the Test Data?
• Which version of the Test Data I should be using?
8. What is the threshold I'm targeting with Test Data?
• What should be the size of the data in DB when I make the IO and RW operations?
• What should be the network capability when I make the IO and RW operations?
• What should be the hardware capability when I make the IO and RW operations?
• What should be the geographical traffic and its pattern when I make the IO and RW operations?
• More of such factors will be considered when identifying and deriving the test data.
9. What is the client error yielding Test Data that I should have for the workflow?
10. What is the server error yielding Test Data that I should have for the workflow?
11. What is the redirection yielding Test Data that I should have for the workflow?
12. What is the no-response and no-change Test Data that I should have for the workflow?

And, more.  It is simple; get in touch to discuss and know beyond the listed.

To conclude and stop here, all these questions, do not ensure or assure or make sure that I will have test data for evaluating a characteristic of performance.

• It helps me to know:
• What are the tests I should be doing?
• What kind of preparation I should be having in my practice to create the Test Data for these tests?

The, Test Data should challenge the available Testability and its limits.  If it is not doing, then, we are having a test data no doubt about it; but, it is of shallow. Shallow!?

One has to ask self, "Is this sufficient enough and effective Test Data for the system [and workflow] I'm testing?"

The, Test Data should drive the engineering team to add more layers of Testability into the system.

Test Data is Not [Equal To] a Test

 

Not every release is a critical release!

There are releases which are critical from technical and business interests.  In such critical releases, most of my time is spent on understanding the technicalities of the system and the test data identification on identifying what are the priority tests.  Identify and building the test data takes major chunk of the testing time. This effort has helped me, testing, stakeholders, projects and products immensely.

Test Data != Test

Looks like the word "Test Data Management" has become a buzz and marketing word.  In how the word "Test Data Management" is pushed, to me it looks like the intent is missing for -- how to identify the tests and its test data?

• If I cannot think of tests, how can I think of Test Data?
• How can I think of tests, the right tests in the context, and the priority tests out of it? 
• If I cannot get this, how can I get the Test Data for all these tests?

Before talking of Test Data Management, we need to talk about the intent and goal of the test.  If I have a clarity here, by being aware of the product's technical internals, architecture and its eco system, I can think of test data in a better sense.  For first, the PaaS which provides Test Data Management solution has to push and enforce for learning and mentioning -- What is the intent of the test?

When I know, for what to evaluate and how should I be evaluating it, those test intent will lead me to the Test Data and its dimensions which should be vectoring the tests.  

Test Data is not [equal to] a Test.  Test Data is one of the variance, in fact, it is a multi-dimensioned variance that a test will [need to] have.  While a test has one deterministic objective, the test data will have multi-dimensioned vectors that instruments a test.

Test Data has its role in a test, and, it is a critical role.  Hence, we practicing Test Engineers talk and spend our time on Test Data in equal importance as we give our time to identifying, designing and execution of tests.

Test Data is not a test; but the test data is an expression of the test's intent.  Test Data is one of the byproducts of a test.  Having this clarity is important.

Note this, the test data is not the only expression of the test's intent.  A test will have multiple touch points; these touch points express, advocate and can show the intent of a test.  A test data is also a heuristic as a test is.

 

Test Data Management

The word "management" is underrated and not attempted to understand from context where it is being spoken.  How do this word sound -- "Test Data Leadership"?

In engineering, especially in the software engineering, the word "management" inherently talks primarily about design.  On day-to-day operation, the management designs its strategy and approach to solve the problem and challenges.  

When said "Test Data Management" in software engineering, it is about strategizing and approaching the problem of identifying and categorizing (subsets) the data to test.  A kind of leadership at one layer.  It has its role and critical to the deterministic outcome for a test.

In machine learning, we can see this categorization -- training data and test data.  Then, is training data not a test data?  It is!  The training data is designed [managed] to train and this data as well identifies the problem as the training progresses.

The data which are identified, designed, categorized and collected as Test Data, are well sampled data in the context, for the intent of a test.

To conclude, Test Data Management is one of the correlation in software engineering to solve a problem; it is contextual in how and what tests and testing is executed.  The different teams and organizations has their own way of managing the test data.  How they do it, it is their way of doing it.  We have to look Test Data Management from point of Test Engineering and not as an another buzz word to sell just as a product and business's service.

Who is a Developer?

 

Who is a Developer? 

I understand the developer is the one who builds.

• A product owner builds the product by bridging the gap between market, business and consumer.
• A programmers builds the product by programming.
• A tester builds the product by testing it with her/his tests and programming the test code.
• A devops engineer builds the product by building and managing the pipelines and environment.
• A DBA builds the product by writing and managing the database environments.\
• A support engineer or executive builds the product by troubleshooting and listening to the problems being reported.

Who else do you see in developing the software system and its product as a solution?

If you see,

• With the skills and expertise we have and build consistently,
• Each of us will work with our expertise in a particular space of the problem solving as a team and business.

Each of us develop some kind of artefact that are combined to build and ship a usable software.  To me we all are developers with a specific roles.  As a developer we upskill consistently and catchup with current and upcoming changes.

Note: The above said are not the only tasks of these roles.  Each do beyond this in developing & continuous delivery of a usable software.

The Agile & Testing: We're Pulled & Hidden from "Shift Right"

Do I Understand the Agile?

If I work in a project which follows the Waterfall approach, I still can apply the Agile Manifesto and the 12 principles behind it.

But, am I adaptable to the change and uncertainty for better ways of developing software and helps others do it?  This shows the mindset of me and others involved.  Have you heard this line, "every project that claims Agile, it has [a small] waterfall in it?".

Agile Software Development is not wrong!  It has values in what it says and for what it stands.  I have no doubts in it!  It describes how I perform a activity and respond to a situation in the software development.

Frameworks and Methodology

We have picked the frameworks which are under the Agile's umbrella. We pick the methodology that suits our work and have customized it to our contexts.

This is what Agile also say, the team will always need to adapt its use of a framework to fit properly in its context.  

Dr. Alistair Cockburn, says, the Agile methodologies are the conventions that a team choses to follow in a way that follows Agile values and principles.

The Agile is Not a Problem!

We do not collaborate well in solving the problem and say Agile is not working and creating more problems.

In reality, how we respond to the uncertainties and what got prioritized, it will derail us.  We derail from what we should be achieving as team [and as an individual] in a given time frame and context.

The Agile as I Understand

In a nutshell, the Agile is about

• Balancing effectively with collaboration in the context we are.
• Identifying, learning and solving the problem as a team no matter where we are in the project's timeline.
• Being ready to face the uncertainty and responding to the changes.
• See the progress and value in what we do as a team and bring the "continuous" in practice.
• Having the consistency in the excellence of our work and the value we deliver with it.

Read the Agile Manifesto and its 12 principles.  It is the day-to-day life of a team who works to build a software in a continuous delivery having the values to a stakeholder.

Testing and the Agile

Testing is a collaborative activity.

The Agile also talks about collaboration as an essence and necessity in the continuous delivery of the software we develop.  

The Agile talks about short interval consistent deliveries.

• How should I fit myself as a Test Engineer here?
• How should I upskill myself to deliver in short intervals as part of continuous delivery?  

I ask and say to my test teams,

• Think of delivering value consistently in the shorter intervals.
• Rather saying -- releases in short intervals, call it out as a value to a stakeholder in regular short intervals as a continuous delivery
• It sounds positive, affirmative and profound.
• Upskill and help your peers to build the skills. List the skills, let us discuss and start our practice on it.
• This mindset changes the perception in how you approach the testing.
• In these short intervals,
• Let us prevent bugs in the ways we can think of.
• Finding bugs also need resources.
• Let's start investing in building and improvising the resources to prevent bugs.
• How can we prevent and where to begin?
• For what should we test and automate other than functionality in these short intervals?
• What is the priority?
• If I do not test and automate for evaluating any particular quality criteria, does it impact the value to a customer in what we deliver?
• If I do not test and automate on a particular seam and system's boundary interface, does it impact the value to a customer in what we deliver?
• How am I collaborating with stakeholders?
• Who are my stakeholders?

In short, I see the Software Test Engineering practice in the Agile setup as this,

• We [team] exploring technically and analytically;
• Evaluating collaboratively in the software development and engineering, by having the effective staged feedback loops in place to improvise the delivery;
• By responding actively to the changes and uncertainties that comes anytime in the timeline of a delivery and practice;
• Thus, the consistent continuous delivery is delivered with value to the stakeholders.

Note: This is my observations so far having worked with 46+ startups in my software test engineering practice.

We're Pulled and Hidden From - Shift Right

I and you are said to Shift Left and start the testing from the inception of a feature's discussion.  And, further we're said, we have to shift left from right.

Did the Agile manifesto and its 12 principles say explicitly to Shift Left from Right?  Who said that?

What I see, How I see and understand the Right and Left here, matters a lot!  If I see the Right within the delivery loop, I'm setting a trap to myself to be late and lag back.  And, probably this trap gets set in the work, that is, to see the Right in the [continuous] delivery loop.  At certain context in continuous delivery loop, starting from Right is a need and necessity.

I see the Right outside the [continuous] delivery loop, as well.  In this Right, I see:

• What is coming up in the tech stack and technology?
• How this is a need?
• How this [is to be and] will be adapted by tech solutions we deliver?
• How should I upskill myself here so that I fit into the loop of continuous delivery?
• In short, the Right should be observing the trend and its analysis so that I'm aligned being agile.
• I will be skilled to test and automate in this change and uncertainties.

The Agile never said to completely move or step out of the Right.  Or, I have not read it in its manifesto and principles. 

You see and learn, what is your Right and ask am I aligning to it for being agile in the Agile.  After all, the Agile says adapting to the changes and uncertainties, yet delivering the value to the stakeholders consistently in short intervals.

If seen closely, the Agile is also about balancing, isn't it?  If I'm moving towards Left and me standing in the Left, am I balanced?

In short, we are being pulled and hidden from Shift Right.

Shifting Right is important to see what is outside the delivery loop.   Are you seeing your Right?  If yes, what is the trend analysis you see?  Where you need to upskill?

Shift Right can also be to look what you can bring into your Shift Left for better development and delivery.  For example, what you see in the market and business space for your product?  Do I have anything to add now in Left from this learning?  Do you see that?  Are not we missing this when we pulled and hidden from Shift Right?

Left or Right, if I do not respond, adapt and deliver for the changes, I need to ask, -- Am I agile and testing in the Agile?

To conclude, know what is your Shift Right while you are delivering values in your Shift Left.  Strike the balance and ace.