Monday, September 16, 2019

Software Testing as I see - A decade back and today, here is a start to the story of reality



This blog post shares insights of a tester who have crossed a decade, doing testing and practicing the testing.  If you are also a decade young, or bit more younger, or a tester who got into testing in recent times, you should be reading this.  If you are not a tester but you work with tester or the testers report to you or there are testers in your organization, then you should be reading this blog post.

It was around 2006, when I got into testing.  Among the standard questions, these question was common in the interview then -- explain the SDLC and explain the V model.  Forget the word Agile, it was not even relevant then.  By 2012 i.e. the word Agile started to appear as a question in the interview.

A decade back the internet was not strong as today.  Today, we have information written and web can host it.  Then, there was no community visibility as today, though it existed.  Internet was not available to all then, you won't believe this.  Today we carry internet in our pockets and palm.  

Most who hired a tester, would have asked to write test case and ask how test would be, in interview. The question around QTP, Load Runner and Cross Browser Testing.  Further a case asking the difference between sanity and smoke.  If went ahead, it was how would you test a pen, that was the question.  Then the tester was said -- you don't have to code; you don't have to learn programming; you have to test, that's big enough task.  Today, the story has changed -- we hear you to have code and not test, for a tester.  Don't you agree?  You have not been said this today?  Or did someone from your friend did not share this with you, that what happened to her or him?


If stood above all this and looked, there was one missing element.  The Software Industry then did not know how to assist a Software Tester to build her/his career.  So the management and peers, did not have much idea on how to assist a tester in her/his team.  That have come in generations passed on and as my seniors went on and got it, I too received it.

If there was one exposed and relevant tester who had vision for today's need, probably the shaping of me would have been so different.  I have tried my best and trying my best, when I learned this isn't what I should be doing as a tester.  



I know in 2019, still there are management and team, which do not know how to help tester in their organization and team.  It is no change, I see there.  But there are few companies which have shown the changes, but it is invisible when seen a picture as a whole.  All the buzzword -- Agile, Sprint, Backlogs, Automation, x Testing, CI & CD, DevOps, xDD etc., have come and striving to remain.  Yet there is no much change in life of a Tester in the Industry.   I had the same thought then as well and today as well.  Then, why didn't I solve it and just saying i.e. writing here?

I have done my best to the team and to organization with which I have worked.  I have to align to vision and goals of the organization as well, while I say the testing can be this today.  Change in the organization and industry, is not so easy.  But I see, it is a ripple, someone did there and when it ripples, the industry i.e. different organization want to ripple the same.  Note that the culture and practice who did it initially, had it so much into it; but the one followed the ripple just imitated it and imitating it, without knowing, without understanding and by not practicing like the one who did and found the value out of it.  This created the difference and misunderstanding, in my opinion.  And this is continuing today as well.  May be it will continue.  This is not a worth problem to solve in my opinion.  Because the delusion of ripple cannot be changed to who see it and want to do same.  Then, what's the point in reading further in this writing, right? Wait!


I see testers asking what should I do to programmers and managers despite they are testing for 'n' years.  Do a chef in hotel and mother in home ask how should I cook and can I do it this way?  Won't we wait for their cooking and eat?  Then, why the testers ask their managers/programmers should I do this or not -- while manager has no idea of testing as the tester has?  Learning from manager/programmer by asking is good; but not how to test and what to do after certain point of time.  I understand, we should inform the manager and stakeholders on what we are doing and why, but not asking what to do.  If asked that probably the manager should get additional (part of salary) salary of tester as well.  If the manager is not getting that additional salary, that manager is being cheated miserably.  This problem did exist a decade back, two decades back, exist today too, it will be there for in future as well, if a tester don't understand testing and practice testing.  If I'm a manager, I will in-turn ask "what you want to do, you do, just let me know in brief before you do that; if there is anything I want to say on that, I will.".  Further, I will ask, "how can I help you?" to testers.  As a manager, I will be wanting to build testers who decide what they should be in their job as a practitioners; I do not want to be instructing as steps in the test cases, step-1 do this, step-2 do this, and this is expected result and asking what is your result? It is our result, when I assist them to grow! You can read it again, the last 5 sentences in this paragraph and it is worth, I see.  

On the upper view like a view from the eagle eyes (see it is so clear), the testing management most times was in hands of people who never tested and it is continuing to be so.  Could be this is one of the primary reason why the testing takes a big hit most times and the testing practice goes to below par each time.  What could be done here?  As a tester, I feel, I should be assisting the testers who are interested and determined to be a value adding tester.  I don't want to change management mindset or testers mindset.  I want to assist a tester by pairing up in practice, that's all.  If I can do that with just fewest testers, that's enough, I feel.  Those testers will do much better when they see their practice and what it has to be.  I just want to pair up and practice with testers in their mindset and not in my mindset; this is easier than working to change the industry and management.  I see the tester as a velocity; while the industry and organization is the acceleration.  To see a better picture of acceleration, velocity has to be clear in its deeds.  Hey, I have a fundamental problem topic i.e. tester not getting the thoughts of business and management. I will write it under a different series of post, soon.  This fundamental problem has created much bigger problems than management and industry not understanding the testing. You know why, because tester did not understand manager, business, organization and industry.

In the next continued post of this series, I will share about the testing practice assistance that was received a decade back and today.  Till then, if you disagree or agree or neither of this, do let me know, if you wish.


Wednesday, August 28, 2019

App Crash! Testing around and inside the crash



In day and out, I come across testers, programmers, managers, and management having efforts on fixing all the crashes. Yes, all the crashes.  In a way I see, if the app did not crash, I will not know the areas that is not being handled well enough.  My testing focus areas will also have tasks noted in such areas to test and learn as much as possible. I do that task provided I can make/given time for it as it is unplanned task.



The common checks to handle crash!?

I learn, an exception if unhandled at runtime, it leads to crash.  There are multiple exception that app can witness which we never thought of during development.  In my initial days of testing, I was in assumption, if we can have -- null checks, index check, illegal argument check, and state check for an activity, we have handled most of the exceptions.  I learned, I'm wrong! How many checks can I write in the code.  I'm not a programmer by job.  I'm a tester. 

I see these checks are not enough and few more got added to my test strategies eventually -- race conditions, unexpected data, wrong data, environment factors, and many more. The collection of these checks is continuing to grow.  Do I cover all these possible crash inducer collections in my testing?  No, I can't and I won't have that luxury of time as well.  Technically, I will learn and prioritize what to use and when.



How the check looks in code, to me?

I write code for automation, which I need to assist my testing.  Here I did write such checks.  At a point of time, I saw, the automation code was full of checks.  Is that a right way?  Definitely not!  A professional and skilled programmer might not do that.  If a programmer has to have such check in each layers of the app architecture, will that sound good?  Personally as a tester, I will not design my tests that way.  As I'm not a programmer, I'm not aware of the pros and cons of doing so. At least I know that is not a better practice to have checks in each layer of architecture of the app.


By handling the exception in my automation code, I print stacktrace of exception.  But will I learn from it to be a better tester?  That's the question I have asked and continue to ask myself.  The exception fix I'm doing, is it stopping me from learning the problem which I have introduced by the actions I'm doing on the app? Is that exception blocking me from learning the underlying problem in my automation code and app?  If yes, then I have a fundamental problem which I have to work upon is my thought.



Why it crashes?

Why at all the app gets into crash? I learn, if the app gets into a state it was not designed for, it will and should crash. As a tester, I will have to learn this state (and such states) at the earliest when I experience crash and on reading the crash stacktrace. I will be happy and not make a fuss about it, if I see a crash for first.  

I learn what is the priority and impact of the crash?  Should I invest my time and test investigate further to provide as much as details to programmer?  Or should I report it with good enough details and continue my testing?  I will answer this question to me.  All I wish is, I don't want the user of the app to experience the crash.  If there is a crash, as a member from development team my intent is to keep it to minimal number having little or no impact.  I see the crash is a great source of learning about my work in the app.

I used to be fussy about crash years back be it on desktop applications, database, web applications and mobile apps.  Now, I have come to point, I love them and it is absolutely okay for an app to crash and it should crash.  What I do post crash in fixing, that tells the bigger story.  In my work, the crashes have made the app better because the team was serious about those crashes.



What on having the crash?

Should the user lose the data what she or he entered on experiencing the crash?  I personally, don't want this to happen for me.  If it happens, I will be annoyed!  That said, how to handle it?  That's something we will have to sit with the programmers and team, for discussion.  

At what point in app, encountering the crash, should we close the app and start over again. At what point in app, it is okay to note the crash and pull the stacktrace, and let continue the user using the app with data entered intact. At what point in app, I should not show UI in view on entering crash and what to show, then resume over safely from there on.  Personally I feel this is a team effort and not just a programmer's effort in making it happen.  



Few testing strategies hack to uncover crash

Here are few things I do and I ask my fellow testers to do when testing mobile apps:

  1. Using the test data which will check the data integrity in app at -- entry point, during processing and post processing.
  2. Identifying the states of the app and passing the invalid states in app at -- entry point, during processing and post processing.
  3. Identify the input which is not from a tester and user. Classify the input on which I don't have control. For example -- the incoming intent; the app responding to APIs (default values, entered values and processed values); the app receiving the response from APIs; the device state; app's activity life cycle state and data/state exchange, and many more as this.
  4. Depending on Android or iOS app, much more strategies can be narrowed down to be specific and work.  At the end, what time I'm left with in the test cycle and what do business want, directs me on what to do.


Debugging and Investigation skills

There are libraries which collects the crash on exception along with other details as -- device info, user info and network info.  I have been with programmers and had difficulty in reproducing the crash and experience it in development environment.  This said, are the logs enough to fix crash?  May be, so we handle the exception and continue the flow of app in runtime.  But did we solve the root problem that caused the crash?  No!  This is where, I feel, the skill of a tester comes in and it is very much needed.

This skill defines me what I'm as a tester and value I bring to the system.