I Do Not See “I don’t see”

I stopped writing the thoughts which I’m penning now, on listening to fellow testers at desk. Listening to them, I’m making note of the words which they are using in their discussion.  Altogether, I’m out from the thoughts of what I was writing and I’m into the thoughts of what the fellow testers are discussing.  I was writing an incident of code review by a tester (by illustrating it with the snippet of the code) and how the tester figured out the case which automation would have asserted as no problem while it is a problem.

Okay, now I hear the tester saying “not a problem while it is a problem i.e. a bug”.  Hearing this, I say myself, “yes could be the bug in the written automation code”.  But for the testers, who happened to agree mutually later in discussion between each others, they came to consent - it is a bug in the product as well.  While this is happening, I see a tester walking in to the bay and joined discussion by saying, “I read, not everything is a bug what I think as a bug.” Well, now I’m out of my thoughts completely into which I was and I growing more curious to listen the fellow testers discussion.

I asked the fellow testers, "What is a bug then? How you infer about it?" I hear them saying, "A bug is a characteristic which annoys the user" and "A bug is a characteristic or happening which threatens the value of product and product business".  The other tester iterated, "not everything is a bug while I think it is a bug."

Now I have changed my writing thoughts from the automation case study topic to this topic and I started to make note of the words as we discussed.  In discussion, I see a thick line of contradiction implicitly in these two lines – “a bug is a characteristic which annoys the user” and “not everything is a bug while think it is a bug”.

A user perceives the experience is annoying to her or him and take it as a bug while using the product. But the product not taking it as a bug, because, it does not see a threat to the value offered from it. Then what is the bug here?  Should I stand with a user perception of experience or the product’s philosophy saying no threat to value offered by me?

Alongside the thick of contradiction here, this is a thin line of thought differentiating between the two statements – a user experiencing annoyance and a product’s value what it intends to give. In other perception, it is, “whose words matter to label it as a bug and having it in the fix list?”

If I use a product and I’m annoyed about a particular thing, is it a bug? Does the product think, my annoyance is threatening the value it is offering to user or me? In business, it is purely a strategic call for having the change in the product or to continue with the same. Still, how do three testers can voice about their experience of saying it is a bug and not so, in reporting it?

The other round of analogy for this – if product is functional then it is usable; but If the usability is bad, does the product is still functional; or, the functionality is also bad in terms of experiencing it? It is a relationship i.e. functionality and usability. So the bug and the experience of the user, is also a relationship. With this, the value offered by the product is also a relationship between the product and a user.

If product’s business does not think it is a bug and also not as an enhancement suggestion to product’s wish list, the annoyance will continue to who is experiencing it. The product believes the value is being delivered with no threat!?

With this, the definition of ‘what is bug’ is highly contextual to what the product’s business say. Knowing this, it helps me while not ignoring a user feeling the annoyance for a reason in using the product.

Hearing to testers now, I brought the fourth thread into the discussion, “everything that I say it is not a bug, not necessarily, it is not a bug”.  At the end, it is product’s business decision after my advocacy on that characteristic behavior.  As I won’t be able to find all the bugs; likewise, I will not be able to understand and identify the bug while it is in front of me and the product’s business. Still the products get shipped and the discussion around “what is a bug?” continues.

Technical, how technical ?

"How technical I have to be here to test and understand what information is expected?" This question changes it's form based on Test Coverage expected for context of testing.

When I ask "what does technical mean here?", I help myself to learn what it is for the context of testing. Like the programmer learning technology to write the code for product, the manager identifying and practicing the new managerial skills required for managing the project, I see what has to be practiced technically for what I'm testing.

The 'Test Coverage' hints me what I have to approach and to what extent I have to navigate in the modeled map of system for testing it. Knowing this, it helps me to learn how skilled I should transform into so I can work along with the technology.

For example, if I had to test the Database System which is non relational, then how should I approach it? Apart data reflecting on commit what else do exist?  Say, the product I'm testing is on non relational database. How will I approach it now to test other than the data reflecting coverage? How technical I have to here now to test? What can I automate here and how? What is the area which requires focus of me than mere checks of automation coverage?

For a programmer, the programming skill is one which she or he practices each time. Apart from practicing programming, the programmer also practices coding for a technology by learning the specifics related to it. While doing this, the challenges come up in writing solution and fixing the problems observed. Likewise, to me for a tester, the testing skill is one aspect which has to be practiced everyday. Besides, I along with technology is also essential by practicing the technical perspectives required which supports and strengthen the testing.

Getting technical is not necessarily programming alone. The orientation of being technical has it's scope for testing. Each element of subject has its technicality.

In changing trend of the industry today, one being technical is becoming necessary as a day move to tomorrow. By reaching in time to market on solving the very appropriate problem of the people, it needs the blend of technical skills along with testing skills.

Log Writing, Helps!

It was initial months of my career on taking the Software Testing as my profession by choice.  I remember I was into 2nd week of this job. In first two days, I was said to read the book of William E Perry, by identifying few chapters by a Project Manager.  On reading, I was supposed to update what I have understood and have to answer the questions he asks.  He came to my desk at 3.45 PM sharp every day with a mug and coffee in it.  He was strict but he helped me a lot to practice. I believe he is the one who gave me hike of 3000 INR and it was my first hike and my salary came to 9000 INR. To add up on this, my parents had said me, 'There will be seniors and big people in office; don't talk too much or ask back questions. Respect them and make sure you take good name."

I and other tester by name Kanthraja MP worked together in lab. There was a consultant from other company and his contract was about to end. I was said to take task of automation. The next day when I came, I learned, his last day was yesterday. A lab with server slots, huge hardware systems, distributed computing systems, Windows, Linux and OS/2 machines integrated with centralized hardware.  Automation has to start tonight and it continues for whole night.  

I was looking at that tool used for automation. The company had brought it for license. Those were the QTP (initial version), WinRunner and LoadRunner days. This tool was something which I had not heard at all when I asked my friends.

The automation also runs in US lab and it will be controlled from India's lab.  I just ran the script once on one client machine and it failed. I went to Test Lead and said, I will write the program again and will use that for the over day automation run. I requested for one day time and said will resume the automation from tomorrow. I wrote the program and in evening I configured the systems with it. But, the programming team was debugging for Out Of Memory and I could not get machine to run the program. I asked my friend Kanthraja MP to run the script and wait for 30 minutes. If he sees any trouble with the automation in lab, I asked him to call me. I had to collect a BMTC bus pass in Shivajinagara bus stand, I rushed that evening.

I got a call from him saying the program fails to run. I searched for internet browsing center and asked him to mail the error log it to company email-id. Looking at logs, I learned, it was difficult for me to know what's happening. I called him again and said, I'm coming back to office. He was waiting for me and I see a tense face.

It was difficult to know what's going wrong and what's happening. I wanted to know but I did not know how to learn it nor investigate it. Then, I got an idea to print each variable value, state value and the flow where it is.  Recompiled and started the automation again. Got to know the problems and it was fixed in an hour.  Initiated the automation run on both lab and we monitored for an hour. It was going smooth and we left for day.This helped us to figure out Messaging problems implemented via CORBA, memory problems and elements which were causing them, and many other problems. 

The printing of details in the log helped us lot.

I remember it today as well. For all the buggy program I write today, still I use the verbose prints to maximum extent possible in context. It helps me to investigate and learn systems. More over it helps me to learn my buggy program, better, each time.

To remember is, even the log consumes the space on system. If it is consuming much, then tuning the log writing is necessary.  Format and preface content which helps better for context, prefixing it for each line becomes handy to differentiate. With this, I see, the testability and also the learnability of the log will be simpler.

Fragmentations: Heuristic and COP FLUNG GUN -- Part 3

Back in this post, I have listed very few perspectives of 'Communication' factor. But it is beyond what I see or what I have seen in view of programming and testing. There is a point which is usually not considered in this Communication factor. It is device fragmentation.  It is a buzz word and it lies there most times as it is not attempted in Software Testing to understand what actually it is and what is the outcome of it. Is is just the device fragmentation the problem source?

Interestingly the behaviors noticed on a device will not be because of fragmentation in first place. But the buzz words makes it to feel it is because of device fragmentation. Going back a bit, is device fragmented or the OS is fragmented? I see both and every other factors of COP FLUNG GUN will have the influence of fragmentation and anything that comes in near future will undergo the influence of fragmentation. It indicates, this will never come down unless the user needs from technology come down. Wait, what is fragmentation and defragmentation? It is very much essential to understand because the understanding of COP FLUNG GUN factor gets better each time if this is understood each time.

In an Operating System, everything is file which in turn will be instructions and data. If this is a file system, then processing of data which will be represented as a file and will be broken into block of files. This fragmented files might be sequentially placed or may not. If not sequential then problem starts -- where disk read, disk write and wait time, all increases in Operating Context.  To improve the condition here in Operating Context due to fragmentation we initiate activity called Disk Defragmentation. Defragementation is a process of grouping the same logical content file so that they are connected in close groups and as a result the file operations by an Operating System becomes easiest and space on disk/memory and processor can be used better for other needs. If observed, the Disk Defragmentation in Operating System actually helps to overcome the impact of fragmentation in Operating Context environment. Then why cant we defragment on the mobile device?

It is like this, I tear the novel and throw its paper pieces across the streets of a city, randomly. Say, I can recognize these piece of papers on the city roads. This is fragmentation in a perspective. Now how will I defragment it by collecting and place sequentially and keep it connected logically and content wise? There are lot other factors in the city's environment -- sun, dust, heat, rain, wind, cleanliness of city etc which will disturb me to collect them together and make a novel again and not just the street which I use to commute.  The same happens in the mobile device its mobile operating system. 

The mobile device's hardware is different from each other where the same app has to run on it. Likewise the Operating System gets customized for each of this device. To relate, the novel cover which was torn into different shapes is the mobile device and the pages of torn novel are customized mobile OS.  I have to make points on each of this torn devices and OS. How? As these torn pieces miss my context and in the influence of environmental and operating context factors, it is more likely that  my points might fail or show more problems now which I cannot guess nor anticipate easily.

Few Fragmentation Factors in Mobile Device and OS

1. Hardware Specifications -- Input; Processing and Storage; Output; Collaborating Unit and Capability
2. Software Specifications -- Customized OS and its versions; Other software & its versions; Software updates - device software; component software; Anything that is bound to be updated or receive the updates
3. User's Context -- Usage pattern; Environmental factors; User's choice and Locale Settings; Rooting and Jail Breaking
4. Environmental Factors -- Infrastructure; Service Provider; Network
5. Regulations -- Policies of devices; Polices of user location; Polices of software
6. Application installed -- It's internal way of functioning on above identified factors
7. Other unidentified factors -- which always exists as unknown until identified

Ecosystem Fragmentation Not just Device Fragmentation

Now it is obvious that fragmentation is not just the device. Not just the device fragmentation is all. It is also including the user who uses the device and app as well along with unidentified factors. How to defragment these fragmentations? This is the challenge to the mobile technology which keeps changing consistently with the next version of app or mobile software or hardware as it gets updated or rolled out. As a result, does it get outdated quickly leaving the little to applications installed and to the previous model (and versions) of hardware and software? Fragmented!


Fragmentation and COP FLUNG GUN

While I learn fragmentation is everywhere here, I see it is also observed in this heuristic -- COP FLUNG GUN.  Identifying it in first hand is not possible as it is highly contextual while few identity remains generic in most cases.

Further in this blog post series I will learn and mention few generic and as well the context specific fragmentation factors in this heuristic. Interesting will be how the fragmenation of  Communication is while it is not mentioned in previous post. Did I tell myself about the fragmentation of files on mobile OS and devices while I was saying how it is from external factors? That has a role too in contribution to the external factors.

Now, let me allow myself to defragment what I have got here.  It is fragmented and goes unrecognized!



Note: I keep the technical stuffs in simple relativity and examples so I can learn it anytime and share it to others who wishes to know about it. Being technical and non-technical is expressing in simple and to anyone is what I understand for now.

Ignored OS Component Shows Itself

It was a new feature that was pushed into a production in short time - two days. I designed the tests for this and tested the same. And in parallel fellow testers tested it as I wanted their mind to see the short falls of my tests and the risks. All of sudden close to one month after a release, in one noon the test team received High Priority message indicating production problem and it was this feature.

Fellow testers took it up and switched to production environment and noticed it. I asked if it is reproducible consistently. And it was reproducible consistently. The next question was switch the machine and environment of client and boot to production environment and see if it is reproducible.

Now it was not reproducible in few client machines but it is observed in few client machines. In mean time, I observed the logs of servers for around an hour and did not see any change that I'm seeing from last one month.  I pushed the production branch to test environment and ran the sequences and noticed the problem on staging environment as well.  

This was a major clue for me. It indicated, there is a change on client machine which is with users and as well with programmers & testers which shows and don't show this problem. And, the same code.

Few walked to my desk and asked, "Why is this missed and we said to testing team it is a major change and it should be tested thoroughly." I had to say, "Wait, I have no clue for now other than first clue what I have." Testers seated around me were into silence. The, Technical Architect came in to desk immediately and said to all, this is well tested feature and he has tested each line of code of this feature and I'm very confident in it. I thanked him on behalf of testers. Then why the problem now was the question for which I had to say, "Wait for a day, I'm looking into it. It is a problem, it has to be fixed and no other way to live with it for now."

I'm in no mood or state of mind, to tell what is testing and what is quality to people because I can invest the same time in testing and do the useful for product, team and myself. I pick the stage selectively where I have to stand up and talk and when I have to ignore and continue the work.

Below are the questions I asked for the, Technical Architect

1. Did we release other code than what we tested? I heard, "No"
2. Are you sure that no other code commit is done in RC that went to product. I heard, "The tested code."
3. Are you sure that no other packages and utilities are changed in this release. I heard, "Change are the same which test team is informed in labels."
4. As said, I see no change or differences in server and the suspect is on client now. Any thoughts on it? I heard, "The same changes on client interface and no changes there."

I looked into the production release build's code commit label and all looked same and that indicated no code changes in that build which was pushed to production. I confirmed the same to teams. Now, studying the client machines, I started looking at machines which had differences and which did not have then.

There were no minor or major update information in OS and its associated information in client machine. The next information to explore was, to look the problem reproducible in client machine having lower version OS and the latest yet to release.  I went to the Technical Architect, and we had discussion about a suspect. We had silence and we decided to start working on it now.

During the Test Design time, the risk was indicated that can come to product from one component which the product makes use of in the client machine. And this component is part of the client's machine's OS which controls most of the products that will be installed on the client machine.  Reading the beta development changes and bugs of it at time of test design, it was also tested by installing the latest beta but it was still a beta and changes could come in until it's code and architecture design are frozen.

One of the major change for this component was in its architecture and design how it saves data on Client machine. But here, the product could not wait till these component get frozen and it went out by skipping the words of Test Design saying there will be no impact from this to our product.  Test Design's risk list clearly mentioned this but it was said 'acceptable risk' and the cost is known to us and we are agreed to it.

Test team communicated the design of release plan for the same seeing the risk to product but business could not wait for this. The result was observed in one month of time.

Now, the cost was still bearable but not anymore here because the user will not be able use the Client interface anymore that means no service served from server to user and user is blocked.  I continued exploring and it was already 5 hours had passed and I had lead sources but not yet isolated the cause.

Monitoring closely with much more tests, it was isolated that, component of the Client's OS machine is the cause. This was ignored saying it is acceptable earlier though it was mentioned in risk list of Test Design. But the question I got was why not all user is facing this problem and only part of user.

The Client OS vendor is pushing the change of this component to users in batch across the globe and not in one go. That batch of people whose component got updated in their OS, they doing that sequence of operation are getting blocked with the product. And the question is, will all the user do that? Of course, yes and not, it depends on mindset of the user. It was left to business to make informed decision now, ff business sees that user is important, then you make decision what you should be doing.

Learning the change in Client OS's component was not easy and identifying that is the source of problem when it was pushed as an update to Client OS.  The OS did not show any update changes but it was changed. The Test Design happens at every time when testing and it is not that it should happen at beginning. It is an ongoing activity. In this case, I took bit of time since I knew the architecture of product and Client OS communication process as the component future changes were in beta and the impact. Moreover it was a major release and one slight incorrect move can create trouble to users and business.

Reconfirmed the cause. The Client OS component's architecture and the changes it makes to Client interface, is the cause.  This was fixed in product to handle it and it went to production in a day.

The learning I shared to teams and my fellow testers from this Test Investigation

• We design tests always when we test but also doing it by learning the context of changes happening outside the product, it always helps.
• There is no good and right time to design test. Every time is time for designing the tests. In this context, it needed a chunk of time as it was a major change to product in short time. Hence I took time separately out for it.
• We test to learn the change and its impact as well. Learning changes around our product is always useful.
• Knowing the system architecture and supporting platform architecture is an advantage in testing.
• Maintaining the record list of changes in system and associated system is useful no matter if it is a simple or complex software.