AMYQ: Ask Me Your Questions on Test Data -- Session 1

I made an opportunity for myself in this format -- Ask Me Your Questions (AMYQ).  I want to keep it a live interaction as much as possible and I chose a YouTube as an aid.  I'm experimenting it; I will improvise and upskill here it as I move ahead in this.

It is not a AMA.  It is AMYQ format with a topic which I come up listening to community.  In this format, I collaborate and interact with community listening to challenges and problems in their practice and work.  And, working on a solution approach for their context.

I asked the software engineering community for the questions around Test Data here.  I have received a few on LinkedIn and a couple of them in person.  We will be going through them.  

I will share my perspectives and approaches to deal with Test Data on Ask Me Your Questions on Test Data, while I listen to you.  Please join here.

Details of  this AMYQ Session -- 10th March 2025

• Title: Ask Me Your Questions on Test Data
• Date and Time: 10th March 2025, 8:30 PM IST
• Duration: 30 minutes + 10 minutes
• Interaction: Live
• YouTube: https://www.youtube.com/live/cKS71LgwPM0

Questions Received

From Shrini Kulkarni

1. #1 challenge - setting up data in upstream systems to suite test cases that need to be run. There is AUT and there are upstream systems. In a corp setup -- individual teams are setup for each application. Hence getting another team to set some data in other system often encounters lots of manual effort and red-tapism
2. #2 Reserving test data created in AUT or upstream systems for specific team's use so that other teams do not change it.

From Avanti Gada

1. Creating/finding data set to test features built on LLM's. How to test AI tools which were built using LLM's.
• The application are internal to organisation. To take generic example say there is college finder when student searches with certain inputs it looks at Internet and gets all possible options in results. How to ensure the data fetched by LLM are right

From Sukanya Santhanakrishnan

1. What one should keep in mind when the test data is confidential like passwords/person details? How should the system handle this in terms of security?
2. In the AI era, do we rely on LLM generated test data and how much can we believe those? What are the additional steps we need to take after getting LLM's data? 
3. What are the considerations when the applications handling large datasets under high memory usage? 
4. What are your go-to checklist when you start preparing test data?