In general, I understand, the flake means brittle and inconsistent or uneven. Like the flakes one will have as food.
I'm not a native English. It took me years to understand and correlate between the words 'flake' and 'flakiness'.
Test and Flakiness
There are no Flaky Tests. A test is not flaky anytime.
The responses of a system to a test can be flaky. That is, the response is not deterministic and inconsistent to one's expectation and understanding.
In other words, the responses of the system is unreliable to the same action, data, state and other parameters.
This is not a philosophy of testing or test. In my opinion, it is a misunderstanding that is accepted and passed on in the software engineering.
![]() |
| A test is not flaky; the response of a system is flaky. |
Are there no flakiness when testing for services? It exists! That is, the inconsistent behavior of a service for the same set of data, state and other parameters. What is inconsistent in context of a service's response?
Flakiness is not bounded to just GUI It can be with any UI [User Interface]. And, an API is an another user interface which opens up to the bunch of services.
To end,
- It is not Flaky Test.
- It is a flaky response of a system which a test is letting you and me to experience.
- What is a flakiness to one who is testing for security and performance?
- Flakiness is not confined to the UI response or a functional response of a system.
You decide how you should be seeing a test in your suite and label the response of the system you are testing.

This LinkedIn post has interesting points shared as comments.
ReplyDeletehttps://www.linkedin.com/feed/update/urn:li:activity:7411649984992870400
A test never fails.
ReplyDeleteA test never pass.
A test is neither flaky.
A software test is a tunnel or funnel or channel which helps to experience what I cannot touch and sense physically.
--- It is a tunnel or funnel or channel to know my emotions and its response to the behavior/response shown by the binaries and ecosystem around it
What is flaky?
ReplyDelete--- The context of the binaries and its ecosystem making me experience the inconsistent result.
--- But, is result a flaky?
--- No, the result is not flaky.
--- I tie the label 'flaky' to the experience because I expect and have the confidence in saying the behavior and response should be this and nothing else for these actions in the test.
--- This determinstic confidence is what makes me label flaky when it is not matching my expectation.
--- Should the binaries and ecosystem show the same determinsitc response each time?
--- No! If did show same behavior, there is a problem! If it did not show same behavior, there is a problem!
--- Then when it is not a problem?
--- When that flaky behavior of the binaries and ecosystem is not showing any impact to the business and who is paying to the business.
In short, the test is neither pass nor fail nor flaky. A test is a channel to experience, know, learn and help me to decide and act.
ReplyDeleteThat said, the test never breaks itself nor the binaries. The potential and capable test can uncover what is broken and the broken binaries and its ecosystem.
If something is flipping, the test should highlight it.
ReplyDeleteIf the test is flipping, then the needed and must parameters in the test are missing for the context. I will have to figure it out.
After all this, I asked myself, in the time of AI should I be talking about the word Flaky Test. Should I?
ReplyDelete