Wednesday, December 8, 2010

Why the 'Garage'?

I got a question while talking to one my friend and colleague, why the 'Garage' and 'Testing Garage'? I was thrilled hearing that question. Four years back, I asked for meaning of a word and why this product is called by that word. And that product was in market for almost 3 years i.e. before I started my professional testing career. Today it is getting sold successfully probably bringing good revenue. Finally I did not know why the product was called by that word and what it meant. Later I got an opportunity to meet the global technology chief of the organization of that time and asked him the same question. I did not get an answer except saying "It does not have any meaning." which in turn made me to ask "then why that name?” Let me stop this here. The same with software products I have tested so far, most of the time I did not get answer why was it is called by that name. I do not want to assume myself what that name mean without knowing the actual one, and why it is used; you see the testing begins right with name of the software product being tested and knowing its purpose of existence and for whom, with a question "why should I test this?". Hope you know now, why I was thrilled when my friend asked me about word ‘garage’ in Testing Garage.

Back to garage, Testing Garage. My father repaired his cycle and I use to watch sitting beside him. Near to my home, shops usually called 'cycle shops' that repaired, serviced, rented and sold the cycles exists. Still my father use to repair cycle for himself. I use to ask him, are you a mechanic while he removed parts of his cycle. Today he repairs his scooter for himself most of the time and I keep watching it.

Now what my father did was few observations or inferences while riding his cycle or scooter. He felt something unusual; maybe he compared and evaluated the differences. Inferred something is not as expected or unusual from how it was or how it should be. Later he continued to evaluate with experiments then made a decision what actually the problem is that brought in the differences. What he did here was 'Testing' and while he fixed the problems he thought how this may cause difference again. But whatever he did to fix difference, it gave birth to new differences or brought back same differences gradually.

After seeing father repairing the cycle I wanted to see how big vehicles would get repaired. I went to garage with my father which use to service ambassador car and scooter. I saw the mechanics heard the words of customer and started to look into the vehicle asking questions to customer. Again here what mechanic did was not just comparison, but evaluation too.

I made few common observations here. Both my father and mechanic used tools when needed and they used their ears and senses together with replies of other concerned people to know what they are making out. Tools were used when needed and they were spread out on the floor in garage. Father asked me to pedal cycle and use to watch the differences before and after fix. At garage mechanics use to analyze and talk to each other and one use to say this is the problem; probably maybe he was a chief mechanic there at that time. At home father talked to me while repairing the cycle. We shared and my father trusted my words and questioned while I pedaled cycle. In garage, mechanics and customers communicated about the problem by trust with questions to each other and customer who had a concern.

An interesting observation what I made is, father used same tool for different repair and also said that to his friends in repairing their cycle. Looks like I made similar observation with mechanic in garage too. They invented new ways of using the same tool when they needed it. To repair a cycle the tools used were not just made of metal but things like stone, a broom stick, piece of cloth, a rope etc. were used as tools. Garage too had such tools apart from metal tools. Things available around us were used as tools depending on the situation while the cycle was repaired.

I was attracted by this for:
  • Working in pair when required amplified learning, problem identification and solving skills. This was evident for me in garage and repairing a cycle at home.
  • Brought new radical ideas while fixing the problem in the conversational talk. These ideas brought primal changes sometimes which satisfied the customer or concerned person.
  • A skilled mechanic had an objective to know the problem and to fix the problem; focus of mechanic on this was evident.
  • A mechanic evaluates and assesses his or her progress while servicing and make sure that expected is being carried out.
  • I figured out skilled mechanic using strategies with a plan, while trying to know the problem and/or cause of the problem in a vehicle.
  • Mechanic relied upon human skills rather than thinking or saying this cannot be done without a tool. Existing tool or things available around were reinvented as new tools. But tool alone did not do task; a human was needed to do the task and analyze it. Tools helped mechanic to make the job bit simpler.
  • Previous experiences and relative knowledge or information of other repair and problem which is not related to garage or vehicle, was used in evaluation before repair and after repair on comparison using as reference.
  • In garage, reference point or reference source(s) or history of earlier services, played a role in assessing the problem that can come up from service given.
  • At some point the assessment was done just to fix the difference that was seen in vehicle or cycle. But there was no further work or brain bite to know what other problems may persists or may come up, except the checks and tests made for the repair done or service given for then. This was obvious from work and to be noted here.
  • But to identify the existing unusual difference or problem in vehicle or cycle, investigation carried out was interesting to me. This investigation had several tests and not just checks from comparison or reference source.
  • Environment in garage was maintained every day to attend the customers who came up with the (different) problem and different vehicles.
  • Detail of what the problem was and service or fix given to vehicle, was said to customer.
  • In short, mechanic worked technically to provide information and a fix for problem(s) that was known or unknown for concerned person(s).
  • Sometimes decision of mechanic worked. But the final word on quality of work was from customer. Decision at last is in hands of concerned person or customer. Noticed that words of skilled mechanic influenced decision of a customer.
  • Looks like problem or risk was for a person who used a vehicle. But that problem could bring threat to others too on roads. Hence problem cannot be denied with vehicle here as how we cannot deny a problem in software application.
  • In a skilled mechanic, I saw a tester who tried variations by questioning customer to know the problem(s). Though mechanic tested the service given for vehicle, it did not stop problems coming up.
  • In garage, customer defined “Job well done” on using the vehicle which was serviced by mechanic(s).

Relating mechanic at garage with a tester testing software application:
  • Skilled tester will have an objective or goal of testing, while testing.
  • A tester evaluates her or his progress while testing and evaluates that with the objective of testing.
  • A tester makes use of strategies to bring out effective tests that justifies the objective of testing.
  • Being a tester when I worked by pairing with other tester or people, I learned and observed much more. This improved in identifying the problem and issues.
  • The lab, desk and thoughts of tester will have several tools along with tester.
  • I observed testers using tools to help their testing and analyzed the work after using the tool too. This was very same with mechanics in garage.
  • Each tester evaluated response or behavior of software application, based on comparison using a reference source or point. Later application was evaluated to assess probable problems in using it. It was interesting to see how the problems of other fields brought ideas to test software application upon relating them.
  • Lab and test environment was maintained to identify the problems that are not so obvious or easy to identify. Also lab or test machine was maintained to simulate the problems that customer brings out in an overnight.
  • Investigated to find the probable cause of the particular known problem and its symptoms.
  • Tested to assess the problems after fixing the existing problem and its symptoms.
  • Test report gave the coverage and observations made during the testing for people who are concerned about the software application.
  • Tester conducted investigation to assess possible problems underlying or unknown or known for making better decision by concerned people.
  • Decision was made by concerned people and not by tester. Information from tester influenced decision of concerned people.
  • Whatever the test and coverage was achieved with modeling, problems still come up in software.
  • Customer defined how the product is and said “Job well done” by TEAM, on using the software.
Likewise, I can make several relative things in a skilled mechanic working at a garage and a tester testing software. Hence I keep word ‘Garage’ with word ‘Testing’ which reads as ‘Testing Garage’.

Tuesday, October 19, 2010

Why am I unlearning?


To be right I should say "Why should I unlearn?" It's time to analyze and evaluate my skills, learning and mistakes for which I have been accounting so far. Thinking how I can prove myself that my learning is incorrect or not relevant, started to ask questions for myself. These questions showed what all I have learned are just without an appropriate base. I was not convinced with my answers for questions I asked myself. Realized I never worked enough for having elementary things.

OK. What did I learn then? I did not learn much or anything. Just did mistakes every time. I tried not to redo the same mistakes. Unconsciously I was learning here and did not evaluate well enough what I was learning from those mistakes. These mistakes helped me as sign boards for going ahead. Interestingly, these sign boards took me in direction which was awing. Fortunately found few on this way and hardly shared or debated their thoughts. Instead I kept arguing myself on what they thought and did. To be in short, did not learn much in right way. Discovered again, I am a slow person.

Why I did not learn? Still that question haunts me. I have learned lessons now, what happens if I do not learn. What will I be doing now? I'm working on which I do not have by reconstructing my learning model. Why did I write this? I will like to see this as a source that identify problems in me and use it to help myself in solving the problems.

What can happen now? I want to be an informative tester and believe can be. Will take my time and does not want myself to be influenced by any other factors, no matter how late I'm. Anything I want to ask for myself? Yes. How do I know I'm learning? Proving each time I was not very useful to me last time. Anything left? Will update myself very soon and look at my responses.



Saturday, September 4, 2010

While tomato came out, green chili did not.


This post has investigation report on probable cause of problem I witnessed in restaurants with sauce container. The report can be found here. Oracles and Heuristics used to learn probable cause of problem are:
  • Tomato sauce container.
  • Tomato sauce.
  • Size of sauce container's knob.
  • Air circulated around sauce container.
  • Warm water.
  • Allowing blocking contents to dry and time.
  • Dried and wet sauce content.

Monday, June 21, 2010

Reassemble The Bridge: Weekend Testing 38


Mission: To solve the puzzle with help of a partner.


Mental Modeling and Approach:

With the image observed for fraction of a second and URL of puzzle read in chat transcription of WT 38, gave an instinct -- what I saw may be bridge with blue sky and greens.

The scattered images pieces appeared to have different colors and boundary shapes. Few image pieces appeared to have same color but varied in its contrast to my eyes. These pieces had pictures that looked like hanging bridge, moving car, green grass and water. This helped to get a picture in my mind with help of the picture I saw, when I browsed the URL of puzzle.

Approach I used to play puzzle was varying one factor at a time and identifying its patterns.


Playing this puzzle helped me to practice and learn:
  • Observing.
  • Identifying patterns.
  • Factoring.
  • Reasoning.
  • Testing.

Weekend Testing report is here. My report is here.

Note: I did not play as per the mission statement. I played puzzle individually.


Wednesday, March 31, 2010

How safe am I, if I use your application?


Mission:
Find security information which makes the application to loose competition for competitor's similar application.


Application under Test: File Uploading Application.
Session ID: FUA-1
Start Time: 27th March 2010 12:08 PM IST.
End Time: 27th March 2010 3:25 PM IST.
OS: Windows XP SP3.
Browser: Firefox 3.6.2
Anti Virus: Symantec Endpoint Protection.


Context:
Tester has been given just an application release and asked to test it, which is available in market for users. Application owner is expecting information from testing in few hours. The application has demand in market and being purchased by users to upload files to desired FTP sites. In view of the sustenance in competitive market, application owner decides to test for security threats. A tester is assigned with a job to find security loopholes information and to identify how their competitor's similar application withstands the identified security loopholes.


Tester starts learning:
  • What the application is?
  • What platforms does the application support?
  • Why should user this application? What are the benefits of using this application?
  • Who are the users of this application and for such applications?
  • Who the competitors are, with similar applications?
  • How the application is used (or can be used) when available?
  • Why they use this owner's application in particular, when other such similar application are available in market?
  • What type of files can be uploaded? What is the minimum and maximum file size that can be uploaded?
  • When, how and why the application "does not do what it appears to do"?
  • How the application "does what it does" and "should do what it is expected to do"?
  • How the application is built?
  • Any limitation known currently in the way how application appears to work?
  • Any defects reports not fixed or deferred or not released with present build? If yes, why?
  • Do the users of the application are satisfied using it today and earlier? If no, about what they are concerned in using application?
  • Are users looking for new things in application today? if yes, how quick they need it and how they value application if it is available and if it is not available?
  • Does application was cracked for license key earlier? If yes, how frequently the license key or file was cracked or generated? If no, did the team at development site tried to do it so and what was the result? What the decision maker(s) decision when license file was cracked or when it was not possible to crack?
  • Does application server at owner place was hacked earlier? If yes, how frequently it was hacked? What was the cost of these hacks? If no, did the team at development site tried to do it so and what was the result? What was the decision maker(s) decision when server was hacked or when it was not possible to hack?
  • How the sales and revenue for the application has been along with the similar application of competitor's?
  • Does all claims of application are met? If not, which claims are not met and for whom? How bad the cost is to them for not meeting the claims? What claims have been met?
  • Any records or feedbacks from the users of application are maintained? What those records or feedbacks say?
  • What does the customer support feedback say for service being given for application users?

My mistakes in this session:

I was not able to test with all security test ideas I had in test coverage model in given time. Invested time in investigating behavior of application. Need to concentrate and achieve mission in given time by exercising application using test coverage model. Test report can be found here.

I learned whenever we are using anti virus, firewalls, routers, port scanners or any other threat identifying tool, we are using others program code built as an application. These application can have defects within them. The questions that remains now in me is, can an application be easily accessed when unexpected happens to security providing application? If did so, what will it cost the user? If security is understood as freedom from fear, then why many software applications get hacked or cracked? Do software application have no freedom?


Wednesday, March 17, 2010

European Weekend Testing 09 -- Test and Experience Report


Mission:

You are moving from lovely Europe with measurements based on the metrics system to the US with imperial units. Test Converber v2.2.1(http://www.xyntec.com/converber.htm) for usability in all the situations you may face. Report back test scenarios for usability testing until 4.30pm GMT on bug repository.

Application Under Test: Converber 2.2.1
Actual Event Date: 13th March 2010, 09:00 PM IST
Start Time: 14th March 2010, 09:03 AM IST
End Time: 14th March 2010, 10:15 AM IST
Test Machine: Windows XP SP2
Tester: Ravisuriya


Modeling of context:

I am a common man with elementary school education and a very rare computer user. Working for a mining firm in Europe now transferred to US for few years on contract job. Came to know by a colleague that measurement unit changes in Europe and US. With the help of a person in Internet Center browsed for the measurement information. Found few details of Imperial and US customary measurement systems. Later used tool Converber V2.2.1 to note measurement unit I needed.


Understanding of term ‘Usability’:

Usability: How comfort is it for a user to use AUT? How quick can the user learn to use the application for desired purpose? What efforts by the physically challenged people need to put for using the application? Does application supports the Operating Systems accessibility features? What efforts does the user need to put while using it?


Session Report:

I did not participate in the EWT 09; but I practiced it offline on the next day. Report from EWT team is here. Chat transcript of discussion session is here. PDF document of my report is here. The usability scenarios that I identified during the practice session:
  1. User launched the AUT. Was user able to use AUT with the GUI objects available in it when AUT launched? Did the GUI objects help or retarded the speed of using AUT? Was the naming convention of GUI objects was self explanatory to user?
  2. User entered values for a selected unit. How to identify the value seen is of type ‘Imperial’ or ‘US customary unit’?
  3. Users want AUT to have a default option of either ‘Imperial’ or ‘US’ units.
  4. User selected units are same; what message is displayed now?
  5. User not aware whether the calculation of unit value for desired, is correct or not. How to test this by user?
  6. User entered variant value which is not acceptable by the selected unit. How such instances are handled?
  7. User need to convert of area, distance, volume, speed, power, pressure, luminance, temperature and other common used units in industries and daily life. Is there any option to see most common units converted or used across the globe?
  8. Whether all Imperial units are available in AUT for conversion to US customary values?
  9. User feels it was difficult to learn using AUT. Searches for help manual. Is a Help manual available feature, limitations, bugs and contact details?
  10. User wants to enter new units that do not exist in AUT. If added, how the conversion procedure for added units can be added in AUT?
  11. How simple the words and contents are available in AUT, so that user with no much school education can use application with ease learning of units and conversion?
  12. User does not understand English well and can use AUT in Farsi. Does AUT support of Farsi or other languages? Does all the displayed and available contents in AUT are shown in Farsi?
  13. User changed language in AUT to ‘English’. Did the AUT have any Farsi words displayed now?
  14. User did not know much information about the Imperial and US Customary Values. Did the help manual have that information for user to understand them?
  15. User wanted to have print outs of the converted unit values. Does AUT support the print? What are the ways the user can save the converted unit values for using it later? Does AUT support all of these or few among them? Which is most commonly used option to store the converted unit values by user?
  16. User entered the value for a selected unit. The converted unit showed scientific notation. How user can convert that scientific or mathematical notation to values which is understood by user?
  17. User entered a value which was not valid for a chosen unit. Can the user identify the displayed pop up dialog is for invalid value entered and it is from AUT? Closing the dialog will it allow user to continue using the AUT? Does the dialog appear in foreground or in background of AUT? If appeared in background which user cannot identify it, will user be able to continue using AUT? What options are available in displayed dialog to assist user?
  18. User is visually challenged. How the AUT does help user now to know the converted unit or to convert the values into US units from Imperial?

It was a good exercise for me to identify the usability scenarios. At end of this session, had a feel that I did not do well enough in identifying the usability scenarios. When I am finding few more such usability ideas after the session, I could have done better in identifying scenarios; need more such practice.


Weekend Testing 28 -- Test and Experience Report


Mission: There are three tasks to be completed today. Time Duration: 1 hour.

Task 1:
Complete the game: http://www.gamesforthebrain.com/game/dragger/
Objective: Send the screenshot where the picture is built right.

Task 2:
Score 90 points in the game:
http://www.gamesforthebrain.com/game/memocoly/
Send the screenshot. Checkpoints: URL of the game, IQ Score, "Your solution is right, congratulations! (+10 points)"

Task 3:
Score 50 points in the game. http://www.gamesforthebrain.com/game/numberhunt/
Objective: Send the screenshotscreenshot. Checkpoints: URL of the game, IQ Score, "Your answer xx is right, congratulations! (+10 points)"

Date: 13th March 2010, 3:20 PM IST
Machine: Windows SP3
Browser: Firefox 3.5.8
Tester: Ravisuriya

Context: Tester was given mission of playing the game in 60 minutes. Power cut for about 15 minutes before starting the mission tasks. Power came back and had 40 minutes remaining to complete session. Saw testers asking questions to facilitator about the mission and tasks. Started by taking up first task.

To Deliver:
  1. Screenshot of picture built right from Dragger.
  2. Screenshot of score 90, URL and sentence that tells you have scored 10 points from Memocoly game.
  3. Screenshot of score 50, URL and sentence that tells you have scored 10 points from NumberHunt game.

The mission was of confusion to me as it said "complete the game", "Score 50 points" and "Score 90 points". It did not tell why I should score just 50 and 90 points or more. Procedure if any to adhere while playing the games was not mentioned. During the power cut, tried to understand the mission.


Assumptions:
  • Looking at the game description in mission statement, I thought it would be time limited.
  • Tools can be of help here for to accomplish the mission.

Tools that helped me more to accomplish mission:
  • White paper and Pencil.

Tasks:

Task-1) Dragger:

Browsing through the URL given in the mission found jumbled piece of an image. I tried clicking button ‘Refresh’ to find any simple image that I can think of to arrange quickly. Found one jumbled picture that is easy one for me and completed the mission-1.

There was no restriction that these particular image pieces needs to be put in right frame. I thought it was an opportunity for me to choose picture of my interest.


Task-2) Memocoly:

For first couple of tries kept looking at the screen with attention diverted no where. But, this made my eyes strain with no spectacles. I keep the brightness and contrast of being viewed monitor to less than 50. I felt this could affect if I fail to recognize as what color they were. Thinking of how to over come, I devised to use numbering system of regions.

Mental Modeling of 4 parts of a geometric shape that appeared as square:
  • Labeled each section as 1, 2, 3 and 4.
  • Later wrote the numbers on the sheet as per the region of square blinked.
  • Then clicked on those regions based on the numbers I wrote.
  • This helped me to complete the mission in good time.

Task-3) NumberHunt:
  • Used Microsoft calculator to calculate the displayed numbers.
  • This helped me to complete the game-3 bit quicker.

Finally was able to accomplish the mission i.e., to complete three tasks. Among few strategies I wrote for completing the tasks, I used the strategy of using the tool to accomplish mission. The discussion session had interesting discussion and chat transcript can be found here. Facilitator came up with interesting thought for discussion on Weekend Testing forum. The discussion topic title was -- "we are testers (who plan to meet the mission) or testers (who aim to improve our skills)" My report of three tasks as PDF document is here.


Sunday, March 14, 2010

Weekend Testing 27 -- Test and Experience Report


Mission: To generate test ideas to test Google Buzz with quality criteria Performance.

Date: 06th March 2010, 3:05 PM IST.
To Deliver: Report of performance testing ideas for Google Buzz.
Tester: Ravisuriya

Context: Tester has been asked to look for test ideas to test performance quality criteria of Google Buzz. It was an opportunity and first time, a tester was working with performance quality criteria. Google Buzz was live and used by users for buzzing.


Started of with mission statement. I was not very sure what 'performance' meant in mission. With a question to WT session facilitator, assumed 'performance' here as "how quick and responsive it is". Further continued to collect details regarding the context in which 'performance' was considered here. But it remained as open question and was left to tester choice in session. With mental modeling myself as a tester who is looking out performance test ideas for Google Buzz, began to brainstorm.


Assumptions made while brainstorming:
  • Was not sure what the ‘performance’ means in the test mission.
  • A thought, does the performance mean – “how well was Google Buzz was performing among with similar other available social networking services”?
  • Or how well the desired purpose of application satisfies the desired claims with given constraints by processing user actions and its throughput interval.
  • Did assume for this session, the ‘performance’ is how quick and responsive it is.
  • Various users are using Google Buzz at present, with no similar internet connectivity speed.

Few questions that I got while brainstorming about users and Google Buzz performance:
  • Performance test ideas in what context?
  • Who are the users and what kind of users are using Google Buzz?
  • How often they have used and using Google Buzz? What are their observations and perceptions of performance?
  • What devices did they use to browse the Google Buzz? Did this device too have any influence on their perception of Google Buzz performance?
  • How long did they use Google Buzz in a stretch i.e., without any break?
  • What environment did they have or do they have while using Google Buzz? Did it have any influence?
  • Any tools are used to know or understand the performance parameters of application under test?
  • How the performance is being measured? What are the units of measurement of performance in this context?

Brainstorming for Test Ideas:
  • Identify the kind of users, their ages, business, purpose, environment, consistency of using Google Buzz. Few to mention here. This keeps growing as and when brainstorming and testing sessions is on.
  • Identify the possible and potential contexts with each of this environment above said.
  • Usage of analytics and log files of Google Buzz. The log files and analytical information regarding performance of application under test to have time of request processing and of throughput.
  • Test on various possible hardware and *software configurations and network topologies. Note: *software and hardware may be that are installed on device from where Google Buzz is browsed and on Google Buzz server.
  • Does Google Buzz can be accessed by mobile phones? If yes how the configuration and applications can have influence on time factor with Google Buzz responsiveness?
  • Type of database Google Buzz makes use of.
  • Handling the incorrect or invalid entries. How quick the application is responsive to user here?
  • How quick does it handle if I keep buzzing with a very **small interval of gap between two buzz? Note: **small – will be decided on application’s minimum and maximum tolerance value.
  • Does any other application using internet or network will have influence on Google Buzz’s performance?
  • Study of features available or provided by Google Buzz. This might give more ideas how performances tests can be devised.
  • Tests for knowing how the performance of application goes, when number of parallel users increased and decreased, number of transactions in a given time period varied to threshold and least, time consumed to recover from these two edges of variations etc.
  • Performance of application when it's potential reaches to the extreme. What time it takes to handle the requests from clients?
  • Type of server being used. Type of connections. Any data stored by application on client is not stored or stored partially – how the performance of application goes here.
  • How Google applications being used by user simultaneously along with Google Buzz choke up the time. Need to identify such application(s) of Google and need to be tested along with Google Buzz.
  • Other vast applications used over internet and Google Buzz.
  • Maximum requests that can be handled at a given time. (This includes various actions using various features of Google Buzz).
  • Usage of Google Buzz over ***network that is not comfort for using of application. Note: ***network – properties of network.
  • If any tools or program used to look into the performance rating of application, what time is taken by tool to work on this? Does the time taken here by tool or program instructions have any influence?
  • How the request of client is sent to server?

The test ideas I got during this session were of generic to performance. I will use these test ideas to develop specific performance tests. Interaction with Dr. Meeta Prakash showed how these ideas can be turned to be more specific to performance test ideas. Discussion session transcript is here. PDF document of my session report is here.


Tuesday, January 26, 2010

AutonomyV -- Open Source Web Test Automation Tool


Searching for open source test automation tool found AutonomyV among other available. I helped my testing with this tool for automating most repetitive actions in few sessions. Using it was simple for record, modifying script to need level and use the recorded actions in VBScript. Tool can be downloaded from here.

Few limitations of AutonomyV as per my learning for now:
  • Can be used with Internet Explorer only. I made use of this tool with IE7.
  • It supports limited browser objects -- edits, text areas, buttons, check-box, radio button, web links, web images and web lists.
  • While executing the script, the IE window opened will not be maximized. To over come this modified the AutonomyLib.vbs file.
  • Need to execute individual script for each instance of automated actions.

PDF documents "AutonomyV Users Guide.pdf" and "AutonomyV Samples Guide.pdf" available with tool and useful.

A sample VBScript for automating user actions on Mercury Tours demo website is here. And MS Excel file from which data is read can be found here.

Note: Credits to AutonomyV team and Mercury Tours demo website.