AMYQ: Ask Me Your Questions on Test Data -- Session 1

I made an opportunity for myself in this format -- Ask Me Your Questions (AMYQ).  I want to keep it a live interaction as much as possible and I chose a YouTube as an aid.  I'm experimenting it; I will improvise and upskill here it as I move ahead in this.

It is not a AMA.  It is AMYQ format with a topic which I come up listening to community.  In this format, I collaborate and interact with community listening to challenges and problems in their practice and work.  And, working on a solution approach for their context.

I asked the software engineering community for the questions around Test Data here.  I have received a few on LinkedIn and a couple of them in person.  We will be going through them.  

I will share my perspectives and approaches to deal with Test Data on Ask Me Your Questions on Test Data, while I listen to you.  Please join here.

Details of  this AMYQ Session -- 10th March 2025

• Title: Ask Me Your Questions on Test Data
• Date and Time: 10th March 2025, 8:30 PM IST
• Duration: 30 minutes + 10 minutes
• Interaction: Live
• YouTube: https://www.youtube.com/live/cKS71LgwPM0

Questions Received

From Shrini Kulkarni

1. #1 challenge - setting up data in upstream systems to suite test cases that need to be run. There is AUT and there are upstream systems. In a corp setup -- individual teams are setup for each application. Hence getting another team to set some data in other system often encounters lots of manual effort and red-tapism
2. #2 Reserving test data created in AUT or upstream systems for specific team's use so that other teams do not change it.

From Avanti Gada

1. Creating/finding data set to test features built on LLM's. How to test AI tools which were built using LLM's.
• The application are internal to organisation. To take generic example say there is college finder when student searches with certain inputs it looks at Internet and gets all possible options in results. How to ensure the data fetched by LLM are right

From Sukanya Santhanakrishnan

1. What one should keep in mind when the test data is confidential like passwords/person details? How should the system handle this in terms of security?
2. In the AI era, do we rely on LLM generated test data and how much can we believe those? What are the additional steps we need to take after getting LLM's data? 
3. What are the considerations when the applications handling large datasets under high memory usage? 
4. What are your go-to checklist when you start preparing test data?

When Does the Community Choose Not to Respond to Questions?

I read the testing, automation and test engineering related questions posted on the social media and web.  I try to understand the question, problem expressed and collaborate to assist.  I do keep 46 minutes in a day for this activity.  

Sometimes, it is hard to assist reading the question.  I feel like not giving up; but, then I do not see the communication happening actively from other end.

What Makes It Hard To Assist?

The questions asked,

• It will lack the context.
• It does not tell who is the person and what she or he is trying to accomplish.
• It will not have information on
• the environment.
• what is the challenge he or she is facing.
• what she or he tried so far.
• It will not have minimal data as
• screenshot, exception stack trace, the complete error message and details following it, data being used, code outline, and more details to the context.

The above are minimal data needed on removing any sensitive information.  Know and understand what is sensitive information for your context when you are sharing.

Then, what do the question will have?

• Most will have a phrase or a couple or three sentences of what they are doing and what is seen on the screen.  And, asking what to do for what is seen on the screen?

Why it is hard to assist with vague details?

People who want to assist won't have any context about you -- who you are, what you are doing, or why are you doing it that way.  They won't understand your purpose or what you are actually experiencing based on the limited and vague information in the questions.

Without clear details, it is hard to connect the dots or pinpoint the problem.  Instead, people who want to assists will be guessing, making assumptions, and probably considering multiple possibilities.  Is this a way to use the time in community?

When questions follow a similar pattern but vary in challenges and context, it becomes demotivating to decode them.  Over time, people will lose interest in reading unclear questions, leading to fewer responses and missed opportunities for meaningful discussions.

What's happening at other end?

People who want to collaborate and assist will take the time to read the question.  But, when a question is too vague to understand, they often give up.  And, they feel bad for doing so.  The question remains unanswered.

When a solution is provided, there's often no update on whether it worked or not.  Those who contributed keep checking back, only to find no response.  Is that fair to those who invested their time to help?  Would you feel good if you were in their position?

Remember, in the community one can't buy someone's time to listen or solve a problem.  It has to be earned!  And, it has to be earned every single time.

How to Frame and Post the Question?

There is no one excellent way of doing it.  Then, what should I do to post my questions?

1. Know the community.
2. Read through the questions shared earlier in that community.
3. Look at the questions that have found resolutions, acknowledged and accepted.
1. Observe how the question or problem experienced is described.
2. Look at the details shared and how it is shared.
3. Look at the context details shared and how it is shared.
4. Observe how the interactions and conversation is taken forward from the two sides.
1. Closely notice the words and how the energy is kept high in both ends and how each side is pushing for it.
2. Importantly, look at the time taken to respond from both ends.

I do not want to share an example or reference saying this is the way to do it.

You figure out for your problem and to its context.  Share the minimal information said above and ask for the help.  Consistently improvise on how you ask, share and describe the problem.