Do Your Bug Report Annoy You and Fellow Testers?

I read the quotes or thoughts often about the code being written. Like, write the code for other programmer and not just for you; so that, the other programmer can pick it in ease and work from there.  You should have come across similar thoughts on code.

Have you ever come across thought[s] that speak about the bug report being written?

The bug report you write, is it for you alone? Or, is it for the audience to whom you wrote? Or, is it for someone who picks it up and work upon later?

How good are the audience of your bug report on reading it?  How did other fellow testers feel reading your bug report?  How easy it was for you to read your own bug report and work on it later?  How smooth it was for other tester to understand your bug report and test the fix?

I experience this:

• A bug report with a precise and helpful technical details did not serve the audience and fellow testers
• A bug report with no precise and helpful technical details did not serve the audience and fellow testers
• A bug report with plain English and attachments did not serve

While I say, that, I see this helped most audience sometime:

• A bug report in plain English giving the context, little or no technical information and associated details

It has happened, that I have rewritten my bug reports on reading it after an hour.  And, I have rewritten the bug report of others as well after testing the fix.  In both cases, I "prevent" the pain which I and others go through to some extent.  At least, I hope so!

To end, I recall this quote of Martin Fowler

Any fool can write code that a computer can understand. Good Programmers write code that humans can understand.

I see, this holds good for a bug report as well.  All and any of us can write a bug report.  A skilled engineer [and test engineer] can write a bug report which does not bring unwanted pain to her or his audience.

Anytime, did you read your own bug report after 3 months of writing it? How deep was the pain and annoyance to know what it was all about?  Give the same bug report to your fellow tester or programmer or product owner; ask, what did they know from it.  

My First Hand Analysis of CrowdStrike Falcon Update Incident

I attempted to analyze the process dump of CrowdStrike shared by my friend.  He said, there could be an attack which is leading to crash of Windows OS globally.  This made me curious to look into the dump and learn.

I had no much context around it, but, a test engineer in me did not sit quite.  I started to analyze the dump information.  Here is my first hand analysis that I made on 19th July 2024 post 10:30 AM IST.

What I Saw?

• It is a Windows OS's process dump.
• Looks like something with C or C++ application reading how the memory offsets were in the dump.
• It started to read a memory offset.
• Then the process witnessed an exception.
• Here the program could not read further
• Why it could not read further from this offset?
• My little experience of testing drivers on Windows OS for a card printer machine, refreshed and recalled what I had witnessed when testing.

Scratching and Striking My Mind

I started to ask these questions myself while I asked what could have gone wrong.  I could not stop here as I was curious what led Windows machine crash.  I referred to web and learn there was an update by CrowdStrike, and then this incident.

The bugs do exist in every software no matter the level and depth of testing, automation and engineering's excellence.  All software do crash and OS is not an exception to it.  But, what made the update to crash the Windows OS?  Pointing and blaming CrowdStrike or Microsoft is not a way for the practicing test engineer.  If these two organizations are serving its huge customer base, they have something working and reliable.  Engineering does not eliminate problems.

By now, I had a thought that it is not an attack.  It is a software bug!  Where is the bug?  What is the bug?  Was it not experienced in pipeline?

The Open Ended Questions

I had these questions as I analyzed and spoke to my friend.

• What is Falcon?
• What was this update to Falcon?
• How frequently the updates are rolled out?
• How the updates are rolled out globally?
• What pipeline do they have in testing?
• Who is impacted the most in business? Is it Microsoft or CrowdStrike?  Impacted in what way?
• What is CrowdStrike?  What they do?  Who are the customers?
• Where do the CrowdStrike's Falcon sit in the OS and what it does?
• How CrowdStrike works in the machines and what it offers?
• What do the dump say? Relook into it with different perspectives.
• How this could have been prevented?
• How will I prevent this if I join this team knowing this incident?

With these questions, I started to analyze the process dump which was shared.

I had more such questions, but these were the first few that I crossed as I started.

Analysis of Process Dump

My interpretation, tells me the below for today

1. Accept that it is an incident as any other incident which I witness in production environment.
2. Do not fall to the speculation happening around.  Remain calm and focus to interpret and understand your exploration.
3. I see, if it can start to read from an offset and then ending to experience a non-existent or invalid offset, is it a NULL Pointer?
• What is NULL Pointer?
• A NULL Pointer is a pointer that does NOT point to any memory location and hence does not hold the address of any variables.
• If I do not initialize and assign, the pointer will have NULL as its value.
• For example, int *test;
• When I want to access the pointer test (a location in memory) pointing to, I will not be sure what is in the pointer when I read it.
• I may not set it later or set it.
• In this case, the code can tell if the pointer is valid or pointing to a garbage memory
• But, if I declare it like int *test = NULL;
• I can check if was set and initialized
• It is a better practice to assign a NULL value to a pointer during initialization so that we can check if it is NULL or as any address assigned to it.
• This understanding of Pointer makes me think, is it due not initializing a pointer and so the error code c0000005 on reading a memory that is not valid.
• When we assign a NULL value to pointer, it is a null pointer in C++
• We assign null value for testing and asserting
• If the memory is allocated to a pointer or not
• If it has a return address and is a valid one or not
• If a pointer is not initialized, assigning null it prevents problems to certain extent
• With this understanding, I also read, it started to read from an offset 0x9c, and then failing.
• What is 0x9c?
• In Octal it is 234. In Decimal it is 156.
• Can there be such address in a computer's memory? I don't know.
• If it is a access violation, then is it a memory which is in preemption of the OS?
• If so the OS can terminate the program or process which is trying to access it.
• Is this killing the process and aborting the operation of Falcon's IPC and eventually Windows coming to BSOD?
• This tells me it is not a NULL Pointer in first case but not initializing a pointer to NULL.
• I infer, if the pointer was assigned to NULL, that is initialized, there could have been some hint in the state and event when accessing the memory.
• This is my analysis; but, I have not seen the test code nor aware of the product.  All this inference is based on the process dump and my experience of testing drivers.
• It got something in between from update (a config or pattern?) for which it cannot find and read in the memory?  Why?
• This indicates me, it could be a bug, that is, a logical problem.  This is my hunch for today!
4. Data in the dump
• Exception Address
• Read from Address 0x9c
• Exception Code: c0000005 (Access violation)

Testing my Interpretations

CrowdStrike as an org when it caters its SAAS to such a customer base, won't it have a testing pipeline

• It will have, I have no doubt in it.  They test and roll out the updates, I believe in it.

Did they witness any such incidents earlier?

• I searched on web for it and I did not find something similar on the Windows, earlier.

Is this a NULL Pointer?  Are you sure?

• No, I'm not sure.  But, there is something that is leading it to address which does not exist or which is invalid?  I will have to wait for their RCA to know technically what caused this.  But this is my understanding reading the dump.

How do you think it is a memory access problem?

• The error code 0xc0000005 says that.
• I referred to driver easy website for the information because my experience of testing the drivers for Windows OS and experiencing such incidents led me there.  This is what I learn:
• https://www.drivereasy.com/knowledge/solved-how-to-fix-0xc0000005-error/

Do you think the programmer would not have handled the obvious Pointer and NULL initialization?

• I believe there will be a check for Pointer and what it is pointing to.  But is it due to no initialization?  Technically this has to be analyzed which I cannot do.  I will have to wait for CrowdStrike team to share the tech details.

Is this a driver problem that killed the Windows kernel?

• I don't know.  But, the .sys file will not have driver as per my learning.  It will have information about the drivers and any configurations.
• This incident is a problem, which impacted both CrowdStrike and Microsoft.  Maybe, both will have their areas to look and fix it they see so.  But, in this context, CrowdStrike can fix it quicker and that is much better -- is what I understand.
• I'm a Windows user for long time.  I see, Windows has worked well to all my contexts so far.  The Engineers of Windows OS knows better than me here.  I'm not well aware and informed as they are.
• CrowdStrike's engineering team are skilled and they are rolling out updates often in a day.  They have a better pipeline when this is being done.
• But, the question I have is, how did this happen?
• No one lets such problem into production when they are aware of it.  Do you?
• There is something that has not come to their observation and experience.  What is that?
• Knowing this will help to prevent this and similar incidents happening in future.
• I'm waiting to know what did not come to their experience and led to this incident.

What could be in the .sys file of CrowdStrike?

• I don't know!  I want to learn that.
• But, from my testing of .sys file and drivers on Windows OS, I learn there could be a configuration details with certain pattern or information to capture at run time, and help the installed software to run.  This is my learning and awareness from my testing.
• That said, testing at OS level and Anti Virus engines are not obvious.  Testing of drivers is like the risky mines.  What is sufficient and good enough in test coverage?  It needs an expertise at OS internals level.
• Windows OS having such a fragmentation in its versions, updates and patches, it is a battle field and mines for engineers building such solutions for sure!
• I learn, the Windows OS stopped when an application tried to access the invalid region or non-existent memory.
• The update which was rolled out, did it have a configuration or a pattern that showed a logical problem when processing it?
• I have such questions and thoughts that are striking my mind as I think and build a problem model for the same.

Is this a race condition incident?

• I see, it is not a race condition incident as users across globe experienced it.

Is this specific to a Falcon version, OS version and hardware?

• Not all host machines would be on latest version of Falcon, is my presumption.
• At least, n-1 and n-2 versions should be on host machine which experienced this behavior.
• So it is not a Falcon version specific, I see.
• It looks to me as it is not specific to the Windows OS version and hardware configuration.
• It is an application software problem which occurred at driver level is what I see.
• This is an IPC communication and process is my understanding.
• The driver can receive the IPC communication in continuous mode.
• At times, this can get queued based on the application and what it does.

Where is the Problem?

Well, I'm looking and pulling from my visualization by relating with my experience of testing the driver on Windows OS.  I don't know the exact reason or close enough to tell what could have gone wrong.

Reading the process dump, it says accessing a memory that does not exist or corrupted.  One of the high possibility is, the starting offset is seen but it is not helping when reading.

• For example, Ravi has the address of India's Prime Minister house.
• But, he does not know from where to start despite having the address.
• He is void and null in knowing where to start and what to do when he is not initialized with the start location to begin the travel to the Prime Minister's house.
• In short, he do not know where the address is pointing to and what it has, though he is given a address to start.
• Can he access the Prime Minister's house premise without any access granted and authorized to do so?
• If not, won't he be arrested by police or other security forces and stop him?

Do I Know the Precise Problem?

I don't know!  I do not know the CrowdStrike product and platform.  I'm waiting to read the technical details from Crowd Strike.

I see, it comes to the data, state and event.  I would focus on how to prevent it learning which data, state and event led to this behavior.  I think of figuring out the Test Design and Strategy that can help me to identify such use cases.  I focus here and see can it brought into the automation so that it gets exercised and regressed consistently.

If it is due to the memory access that had a problem, I did such tests when testing driver for a hardware machine on Windows OS.  I will share the tests that I did in upcoming blog.

I wrote the technical analysis from process dump to CrowdStrike and Microsoft.  I did not get a response.  Anyways, I'm sharing the overall information in a non-technical way so that it is consumable to most readers here.

Note: Here are another threads of me sharing my thoughts on same:

1. https://x.com/testingGarage/status/1814215089525821763?t=XSFdx69ElL0ZmBOcEFrTjg&s=19

2. https://www.linkedin.com/posts/ravisuriya_%3F%3F%3F%3F%3F%3F%3F-%3F%3F%3F%3F%3F%3F%3F%3F%3F%3F%3F-%3F-activity-7221156949445206017-oeRa

Performance Test Report: Between the Effective and Ineffective Reports

 

In this post, I'm picking the thirteenth question from season two of 100 Days of Skilled Testing.

What is an effective way of reporting performance test results and mention some tools you have used in test execution, analysis, and reporting?

I see two questions.  I see it is not a wise attempt to learn these two questions combined as one.  In my opinion, the second question added, it dilutes and make the whole question vague.

What Should a Report Do?

The report should be contextual, compelling, influencing, and targeted to the intended audience to act upon on making a decision.  The software testing report is not an exemption from it.

The performance testing report should know

• Who are its audiences?
• How they read, relate and understand the information?

If this is ignored, the report will not serve the purpose of commissioned testing.  The effectiveness of a report cannot be determined solely on how the stakeholders responds to it.  

On understanding the risks and problems in the system's current capability, mentioned in the report, the stakeholder might not respond with an action to tune the performance aspects.  This could be for multiple factors including that of business.

Note that, a skilled and problem solving engineer understands the business and how it drives.  Just being technically skilled will not help an engineer to grow in a longer run in her or his career.  The system's performance tuning decisions most times will be driven by business.

Did the report persuade the stakeholders with an awareness, mutual understanding and agreement?  The report should drive this conversation.  If not, we have a problem.

On reading the performance testing report, do the stakeholders get an informed awareness on what happened during testing in the present capability of a system's performance criteria?  Do the stakeholders understand and mutually acknowledge how it benefits and costs the business?

This is the foremost value serving expected from a testing report.  If not, I look at how the data and story is presented in the report.

The bottom line is, did we mutually acknowledge, agree and understand on current capability and consequences?  If not, the basic purpose of the report is not met.

Performance Testing Report

The software testing is a high technical activity.   You agree or not to this, but, this is the reality.

Testing for performance is technical investigation activity. It includes the orchestrated study in correlation of 

• hardware, operating system, network, tech stacks & software used in SDLC, architecture, designs, certain decisions, people, business and you - the test engineer.

The fundamental in-depth awareness and knowledge of these areas is essential and a necessity to analyze the performance's aspect.  The performance testing report will show this trait of you as a test engineer.

We have stakeholders who work in technical area and in non-tech area.  How to compile the effective performance testing report?

There is no one way or defined way of writing an effective performance testing report.  Figure out what works in context of your testing to have a effective report knowing - What Should a Report Do?

Outline of Persuading Performance Test Report

It is a technical story telling in a non-technical way with data, pictures, comparison by relating, metaphors, and contemporary history.  I compose the performance testing reports in line with business targets and objectives set.  I provide a metaphors to relate and know the value and cost.

At times, I will have two reports.  I share it with respective stakeholders.

• One with non-technical summary and conclusion
• The other with technical details, analysis and data from investigation

Sometimes, I include the above two reports in one report based on the context.

In overall, this will be in minimum as part of my performance testing report to start.

1. What part of the system is being tested?
2. Why that part of the system is being tested?
3. Mentioning the vague performance requirements gathered from stakeholders.
4. Refining and precising the performance requirements to be specific, contextual and deterministic.
5. Who are the stakeholders of this report?
• What sections the respective stakeholders to refer for the analysis and outcome?
6. Problem statement of the performance testing statement
7. Brief summary of performance testing outcome. [TLDR]
1. What aspect of system's performance is evaluated and why?
2. Brief summary of performance test carried out and outcome.
8. Detailed Report with Technical Details
1. Analysis and Technical Investigation
2. Representation of data which is analyzed
3. Identification of bottlenecks, risks, problems and its symptoms
4. Summary of the test's outcome

You don't have to stick on to one format or a template.  Figure out what works well in your case so that the intent of your tests and outcome is understood by stakeholders.  Give a structure to your report!

The performance testing reports will have metrics, graphs, numbers and proposals.  The presence of metrics, graphs, numbers and the other said, does not make the report effective.  Then, what makes it effective?  When you call it effective?  When you accept it is not effective?  Only, you can figure it out to your context.  I can assist you here; pull me in.

There is no good [effective] report or not good [ineffective] report.  The report is either

• From a team with skills, experience, trained, and practicing
• From a team which is not trained, and, not practicing

Plan to Digg and Explore the Digg Reader

I picked up this exercise posted by Weekend Testing Australia New Zealand (@WTANZ_) in Weekend Testing (@weekendtesting) on 14th Sep 2013.

The mission given to tester in this practice session is as below.

Create a test plan for digg.com and assume you were asked to test the new Digg Reader product. However the catch is: you only have 1 week to test the whole thing. If you only had one week to test the entire product, what would you test?

I had to understand what was the condition of Digg Reader product to proceed further considering the constraint of one week time. On studying what are the changes and risky areas in Digg Reader for that day's version, I came up with master strategy and initial plan to test the product. However, the strategy and plan keeps evolving as I test.

Looking at the context of product and testing need, I feel the functionality needed the tests to start and cover the other tests spanning to different quality criteria. This report is my session notes having the Test Plan for the given mission.

European Weekend Testing 09 -- Test and Experience Report

Mission:

You are moving from lovely Europe with measurements based on the metrics system to the US with imperial units. Test Converber v2.2.1(http://www.xyntec.com/converber.htm) for usability in all the situations you may face. Report back test scenarios for usability testing until 4.30pm GMT on bug repository.

Application Under Test: Converber 2.2.1

Actual Event Date: 13th March 2010, 09:00 PM IST

Start Time: 14th March 2010, 09:03 AM IST

End Time: 14th March 2010, 10:15 AM IST

Test Machine: Windows XP SP2

Tester: Ravisuriya

Modeling of context:

I am a common man with elementary school education and a very rare computer user. Working for a mining firm in Europe now transferred to US for few years on contract job. Came to know by a colleague that measurement unit changes in Europe and US. With the help of a person in Internet Center browsed for the measurement information. Found few details of Imperial and US customary measurement systems. Later used tool Converber V2.2.1 to note measurement unit I needed.

Understanding of term ‘Usability’:

Usability: How comfort is it for a user to use AUT? How quick can the user learn to use the application for desired purpose? What efforts by the physically challenged people need to put for using the application? Does application supports the Operating Systems accessibility features? What efforts does the user need to put while using it?

Session Report:

I did not participate in the EWT 09; but I practiced it offline on the next day. Report from EWT team is here. Chat transcript of discussion session is here. PDF document of my report is here. The usability scenarios that I identified during the practice session:

1. User launched the AUT. Was user able to use AUT with the GUI objects available in it when AUT launched? Did the GUI objects help or retarded the speed of using AUT? Was the naming convention of GUI objects was self explanatory to user?
2. User entered values for a selected unit. How to identify the value seen is of type ‘Imperial’ or ‘US customary unit’?
3. Users want AUT to have a default option of either ‘Imperial’ or ‘US’ units.
4. User selected units are same; what message is displayed now?
5. User not aware whether the calculation of unit value for desired, is correct or not. How to test this by user?
6. User entered variant value which is not acceptable by the selected unit. How such instances are handled?
7. User need to convert of area, distance, volume, speed, power, pressure, luminance, temperature and other common used units in industries and daily life. Is there any option to see most common units converted or used across the globe?
8. Whether all Imperial units are available in AUT for conversion to US customary values?
9. User feels it was difficult to learn using AUT. Searches for help manual. Is a Help manual available feature, limitations, bugs and contact details?
10. User wants to enter new units that do not exist in AUT. If added, how the conversion procedure for added units can be added in AUT?
11. How simple the words and contents are available in AUT, so that user with no much school education can use application with ease learning of units and conversion?
12. User does not understand English well and can use AUT in Farsi. Does AUT support of Farsi or other languages? Does all the displayed and available contents in AUT are shown in Farsi?
13. User changed language in AUT to ‘English’. Did the AUT have any Farsi words displayed now?
14. User did not know much information about the Imperial and US Customary Values. Did the help manual have that information for user to understand them?
15. User wanted to have print outs of the converted unit values. Does AUT support the print? What are the ways the user can save the converted unit values for using it later? Does AUT support all of these or few among them? Which is most commonly used option to store the converted unit values by user?
16. User entered the value for a selected unit. The converted unit showed scientific notation. How user can convert that scientific or mathematical notation to values which is understood by user?
17. User entered a value which was not valid for a chosen unit. Can the user identify the displayed pop up dialog is for invalid value entered and it is from AUT? Closing the dialog will it allow user to continue using the AUT? Does the dialog appear in foreground or in background of AUT? If appeared in background which user cannot identify it, will user be able to continue using AUT? What options are available in displayed dialog to assist user?
18. User is visually challenged. How the AUT does help user now to know the converted unit or to convert the values into US units from Imperial?

It was a good exercise for me to identify the usability scenarios. At end of this session, had a feel that I did not do well enough in identifying the usability scenarios. When I am finding few more such usability ideas after the session, I could have done better in identifying scenarios; need more such practice.

Weekend Testing 28 -- Test and Experience Report

Mission: There are three tasks to be completed today. Time Duration: 1 hour.

Task 1:

Complete the game: http://www.gamesforthebrain.com/game/dragger/

Objective: Send the screenshot where the picture is built right.

Task 2:

Score 90 points in the game:

http://www.gamesforthebrain.com/game/memocoly/

Send the screenshot. Checkpoints: URL of the game, IQ Score, "Your solution is right, congratulations! (+10 points)"

Task 3:

Score 50 points in the game. http://www.gamesforthebrain.com/game/numberhunt/

Objective: Send the screenshotscreenshot. Checkpoints: URL of the game, IQ Score, "Your answer xx is right, congratulations! (+10 points)"

Date: 13th March 2010, 3:20 PM IST

Machine: Windows SP3

Browser: Firefox 3.5.8

Tester: Ravisuriya

Context: Tester was given mission of playing the game in 60 minutes. Power cut for about 15 minutes before starting the mission tasks. Power came back and had 40 minutes remaining to complete session. Saw testers asking questions to facilitator about the mission and tasks. Started by taking up first task.

To Deliver:

1. Screenshot of picture built right from Dragger.
2. Screenshot of score 90, URL and sentence that tells you have scored 10 points from Memocoly game.
3. Screenshot of score 50, URL and sentence that tells you have scored 10 points from NumberHunt game.

The mission was of confusion to me as it said "complete the game", "Score 50 points" and "Score 90 points". It did not tell why I should score just 50 and 90 points or more. Procedure if any to adhere while playing the games was not mentioned. During the power cut, tried to understand the mission.

Assumptions:

• Looking at the game description in mission statement, I thought it would be time limited.
• Tools can be of help here for to accomplish the mission.

Tools that helped me more to accomplish mission:

• White paper and Pencil.

Tasks:

Task-1) Dragger:

Browsing through the URL given in the mission found jumbled piece of an image. I tried clicking button ‘Refresh’ to find any simple image that I can think of to arrange quickly. Found one jumbled picture that is easy one for me and completed the mission-1.

There was no restriction that these particular image pieces needs to be put in right frame. I thought it was an opportunity for me to choose picture of my interest.

Task-2) Memocoly:

For first couple of tries kept looking at the screen with attention diverted no where. But, this made my eyes strain with no spectacles. I keep the brightness and contrast of being viewed monitor to less than 50. I felt this could affect if I fail to recognize as what color they were. Thinking of how to over come, I devised to use numbering system of regions.

Mental Modeling of 4 parts of a geometric shape that appeared as square:

• Labeled each section as 1, 2, 3 and 4.
• Later wrote the numbers on the sheet as per the region of square blinked.
• Then clicked on those regions based on the numbers I wrote.
• This helped me to complete the mission in good time.

Task-3) NumberHunt:

• Used Microsoft calculator to calculate the displayed numbers.
• This helped me to complete the game-3 bit quicker.

Finally was able to accomplish the mission i.e., to complete three tasks. Among few strategies I wrote for completing the tasks, I used the strategy of using the tool to accomplish mission. The discussion session had interesting discussion and chat transcript can be found here. Facilitator came up with interesting thought for discussion on Weekend Testing forum. The discussion topic title was -- "we are testers (who plan to meet the mission) or testers (who aim to improve our skills)" My report of three tasks as PDF document is here.

Weekend Testing 27 -- Test and Experience Report

Mission: To generate test ideas to test Google Buzz with quality criteria Performance.

Date: 06th March 2010, 3:05 PM IST.

To Deliver: Report of performance testing ideas for Google Buzz.

Tester: Ravisuriya

Context: Tester has been asked to look for test ideas to test performance quality criteria of Google Buzz. It was an opportunity and first time, a tester was working with performance quality criteria. Google Buzz was live and used by users for buzzing.

Started of with mission statement. I was not very sure what 'performance' meant in mission. With a question to WT session facilitator, assumed 'performance' here as "how quick and responsive it is". Further continued to collect details regarding the context in which 'performance' was considered here. But it remained as open question and was left to tester choice in session. With mental modeling myself as a tester who is looking out performance test ideas for Google Buzz, began to brainstorm.

Assumptions made while brainstorming:

• Was not sure what the ‘performance’ means in the test mission.
• A thought, does the performance mean – “how well was Google Buzz was performing among with similar other available social networking services”?
• Or how well the desired purpose of application satisfies the desired claims with given constraints by processing user actions and its throughput interval.
• Did assume for this session, the ‘performance’ is how quick and responsive it is.
• Various users are using Google Buzz at present, with no similar internet connectivity speed.

Few questions that I got while brainstorming about users and Google Buzz performance:

• Performance test ideas in what context?
• Who are the users and what kind of users are using Google Buzz?
• How often they have used and using Google Buzz? What are their observations and perceptions of performance?
• What devices did they use to browse the Google Buzz? Did this device too have any influence on their perception of Google Buzz performance?
• How long did they use Google Buzz in a stretch i.e., without any break?
• What environment did they have or do they have while using Google Buzz? Did it have any influence?
• Any tools are used to know or understand the performance parameters of application under test?
• How the performance is being measured? What are the units of measurement of performance in this context?

Brainstorming for Test Ideas:

• Identify the kind of users, their ages, business, purpose, environment, consistency of using Google Buzz. Few to mention here. This keeps growing as and when brainstorming and testing sessions is on.
• Identify the possible and potential contexts with each of this environment above said.
• Usage of analytics and log files of Google Buzz. The log files and analytical information regarding performance of application under test to have time of request processing and of throughput.
• Test on various possible hardware and *software configurations and network topologies. Note: *software and hardware may be that are installed on device from where Google Buzz is browsed and on Google Buzz server.
• Does Google Buzz can be accessed by mobile phones? If yes how the configuration and applications can have influence on time factor with Google Buzz responsiveness?
• Type of database Google Buzz makes use of.
• Handling the incorrect or invalid entries. How quick the application is responsive to user here?
• How quick does it handle if I keep buzzing with a very **small interval of gap between two buzz? Note: **small – will be decided on application’s minimum and maximum tolerance value.
• Does any other application using internet or network will have influence on Google Buzz’s performance?
• Study of features available or provided by Google Buzz. This might give more ideas how performances tests can be devised.
• Tests for knowing how the performance of application goes, when number of parallel users increased and decreased, number of transactions in a given time period varied to threshold and least, time consumed to recover from these two edges of variations etc.
• Performance of application when it's potential reaches to the extreme. What time it takes to handle the requests from clients?
• Type of server being used. Type of connections. Any data stored by application on client is not stored or stored partially – how the performance of application goes here.
• How Google applications being used by user simultaneously along with Google Buzz choke up the time. Need to identify such application(s) of Google and need to be tested along with Google Buzz.
• Other vast applications used over internet and Google Buzz.
• Maximum requests that can be handled at a given time. (This includes various actions using various features of Google Buzz).
• Usage of Google Buzz over ***network that is not comfort for using of application. Note: ***network – properties of network.
• If any tools or program used to look into the performance rating of application, what time is taken by tool to work on this? Does the time taken here by tool or program instructions have any influence?
• How the request of client is sent to server?

The test ideas I got during this session were of generic to performance. I will use these test ideas to develop specific performance tests. Interaction with Dr. Meeta Prakash showed how these ideas can be turned to be more specific to performance test ideas. Discussion session transcript is here. PDF document of my session report is here.