What Structure Does a Test Have?

I was an audience in webinar from Manju Maheswar on Heuristics.  This webinar led me to discussion with Klára Jánová on how the heuristic will help in having structure for problem solving.  Here are my tweets as my discussion.

There is a question from Klára Jánová -- "Thanks! What structure does a test have?"  In my opinion this question goes down to fundamental and philosophical level of Testing.  When spoken in casual, the outcome of a test is seen in binary that is pass or fail.  Is that right or not so right, that's altogether a different topic of discussion.  I will not get into it.  But that binary is associated as result to a test and it has an experience attached to it.  That "experience" attached to the test is an "essence of test" which I would like to bring here in this post.  In simple, if a bug is an experience that I encountered in using the product, then test is an experiment to know what is that experience.  The test exposes a tester or anyone to an experience with information, as an outcome.  How we act upon this experience and information on witnessing it, is what tells the further story.

What we feel out of an experience is the shape or structure we give to it.  That said, the test has a structure to it as an experience. This experience will let us to respond further rationally in a structured and organized way to learn further by debugging.

Apart from the said above, there are much more elements that adds structure to the thought of a test identified. The picture shared below will give the gist of elements which fine tunes a test to be precise, deterministic, practical and an experiment with a question.

Above all these, a test is a heuristic.  That means, an experience is as well a heuristic.  So, the software is a heuristic having sets of experiences to its users.  The software has a structure in multiple forms.  The functions (methods), classes, packages (modules) and the data structures used gives the structure internally to a software.  How the product is built and interfaced gives the external structure.

Now, if I happen to talk in this philosophical tone may be not all take it seriously, is what I believe.  I can understand it and nothing wrong in it.  That’s an experience too!   I will have to communicate and talk how the team with which I work communicates, you see that’s a context.  Usually the interests will be in -- binary that is pass or fail; the artifacts which is by-product of testing activity (test cases, bug reports, etc.); the tools and more.  I will have to work in that mode in those contexts.  A function (method) and class written will have yield to an experience from what it does.  Yet I hope there will someone in programmer, manager and in software engineering, can relate to experience part of a test.  When I talk to testing practitioners who speaks the language as this, I talk to them as this.  That's the context of people and practices, again.

Okay, how's the experiences are structuring for you from the encounters with the tests you executed and from the code you wrote?

HTTP Status Codes and Error Codes: They are not the same!

To write in brief, I understand the HTTP Status Code and Error Code are two different topics when it comes to APIs.  Most often, it is confused that HTTP Status Code is the error code.  No it is not!  Then why it is taken that way?  Could be because of 4xx and 5xx which inform about client error and server error. May be this has lead to assume or take the HTTP Status Code as error code.  If you are taking it that way, fine!  Don't do it from now on and if did, it is not right.  

The 4xx series of HTTP Status Code tell the user (that is client) about the error that occurred from the client input or from the interaction of client.  Likewise, the 5xx series of HTTP Status Code tell the user about the error which occurred at server end when processing the client input or client interaction.  For example, HTTP Status Code 404 in response by server says, the resource being requested by client is not found.  The HTTP Status Code 500 in response by server says, there was an error at server end in processing the input or interaction of client.  For more details about the HTTP Status Code refer here -- https://www.restapitutorial.com/httpstatuscodes.html

Then what is an error code?

Say, you were trying to authenticate yourself to server in a request. The authentication fails and the server returns HTTP Status Code 401, which means unauthorized.  Client when received this HTTP Status Code, it says to user about failure of authentication.

This is not over yet.  Today's microservices are so agile, scalable and adaptive, it can tell clearly what went wrong if well implemented.  Then can't microservices tell why the authentication failed?  It can, if we did implement that.  What was incorrect during the authentication activity?  If this is identifiable and can be said precisely, it helps the user to correct and attempt again to authenticate, right?  This is where the "Error Code" come in handy!

For example, for the HTTP Status Code 401 that is unauthorized, there can be multiple reason. Few reasons as to mention here -- incorrect user account, incorrect password, incorrect auth token, etc.  Now when server responds just by status code 401, will it help?  Yes it will; but can we derive much more precise help?  Of course, we can and it is by defining the error code and it's message for such failures of 401.  Refer below example.

Incorrect User Account HTTP Status Code: 401
Error Code: 1001
Error Message: Invalid user account 

Incorrect Password HTTP Status Code: 401
Error Code: 1002
Error Message: Incorrect password used to authenticate 

Incorrect Auth Token HTTP Status Code: 401
Error Code: 1003
Error Message: Incorrect auth token used in authentication

If you had observed above, all the actions yields back 401 response from server. But to tell precisely what happened, services will make use of defined error code and error messages.  When the client receives this agreed error code message in response as a contract, it displays appropriate message to the user.

Further, client will be programmed with this error code in processing the response from microservices.  Based on HTTP Status Code and Error Code payload received, it acts accordingly.

Here is an example HTTP response with status code 401 and an error code payload:

HTTP/1.1 401 Unauthorized
Content-Type: application/json
Content-Length: 123
Connection: close
Date: Sat, 11 Apr 2012 15:04:31 GMT
Access-Control-Allow-Credentials: true
Access-Control-Allow-Methods: GET, POST, PUT, DELETE, HEAD, OPTIONS, PATCH
Access-Control-Allow-Origin: *
X-Request-Id: Req-87c-96fa-e65e6efcbcde
X-Trans-Id: abcdefghijklmnopqrstuvwxyzn0=
X-Transaction-Id: Txn-41cb7c71-b123-504f-c206-a52d651c

{"code":"1003","status_code":401,"header":"Unauthorised","message":"Invalid access token used"}

The client will look for the HTTP status code in header and in payload, along with the error code and message.



What tests can be done here?

The tests of all quality criteria can be done here. It needs to be well thought, modeled, and designed.  Also, the important test which missed here is the contract test between the client and server for defined error code.