Before Identifying and Listing My Tests

 

I read the below query in TTC's Telegram chat. The discussion had started on this thread and fellow members here were responding.  Further, I read this line and it made me look into it -- "The question was we have to use valid username and password..and perform a negative testcase".

The Default Thinking and Applying Interface

Including me, I see it is subconsciously common for us to approach the problem statement visualization in terms of Graphical User Interface.  When I ask why it is so, maybe it is rooted in our subconscious thinking i.e. with first order and second order or any orders of thinking.

I want to give a try to attempt approaching it by reminding and asking self the below questions:

1. Is it a GUI specific problem?
2. Is it a problem that is tied to the context of GUI?
3. What does this question encapsulate within and open as an interface?
4. What forms do these interfaces take when I stand out of specific interface?
5. Should I stick to one interface to learn and attempt this problem?

Identify the Tests and Framing of Tests

We test to learn

• Does the system do what it is supposed to do and how, why, and when?
• When the system does not do what it is supposed to do and how, why, and when?

Should I call it Negative Tests?  This is not what I share in this post.

To me, these are tests that help me to learn when the system responds and behaves in the other way than I expected.

I can start to identify the straight use cases for inputting an error (a human introduced error) at a given state/data/event; then look for the behavior of the system.  It is good when we can keep identifying and ideating the use cases.  

We get limited with use cases as we continue to think about use cases.  That said, for sure we will identify and frame the tests within identified use cases.  But, we need tests that help to learn when the system fails in doing what is supposed to do.

To supplement it there is another way, which I use.  I do not say this is the only way to supplement.  I use multiple approaches to supplement and identify the tests.  When I do so, I ask the question to the system with the help of these tests and evaluate the response of the system.

Questions to Identify the Priority Tests

I learn and understand the system each time, to identify the better tests.  And, each time I learn something new about the system that I did not know.  

When I'm asked a question in the interview, I ask for details that help me to test better or to demonstrate my deliverable better.  I will watch the questions that I ask!

If I were the candidate who got this question in an interview, I would ask the below questions.  When I learn this is good enough for the initial tests, I will pause with questions.  I move to identify and frame the tests using the responses I got for the questions that I asked.  

These questions will surely help me to be precise and close to the context that better demonstrates my testing skills.  If it is not close, then there is a problem (or a difference) in my presenting and expectations in the interview.  I will have to address it with the help of the interviewer.

Questions:

1. What is the interface where I'm entering the username and password?
• Where is this authentication used?
• On UI (if so which UI), or CLI, or touch interface, or what is its interface type?
  
• At which layer of the system this authentication is used?
2. Where is the format of username and password?
3. What is used as Authorization identity on successful authentication?
• What happens if my authentication is not successful in the UI you want me to test?
• How do I understand that UI is communicating to me that my authentication is not successful?
4. How is this authentication processed?
5. Where the authentication is mapped to authorization and stored for references?
6. What protocol is used to communicate in authentication?
• What protocol and communication order is used to grant and revoke authorization?
7. Who uses this authentication and authorization?
• To know the different means of doing the same
8. Is there any other form of authentication that grants me the authorization?
• Do these different entry points of authentication update my authorization?
• Will I have different authorization data to authenticate? If yes, how the data, states, and events are maintained for my authentication and account?
9. What's the language and Unicode supported by this system?
• Will the languages and Unicode used in the system have any impact when I try to authorize by changing the language and Unicode?  How does the system understand these differences and maintain one state of data with authorization?
10. Are there any computing differences for authentication and authorization on big and small endian machines?  If yes, how and for what context of the system's behavior, processing, and decision?
11. Where and how the authentication and authorization details are processed, stored, and presented back.
• Is there any specific reason for doing it in this particular way?
• How you have strengthened the authentication process to grant the authorization?
• For example, 1FA, 2FA, nFA, what else?
12. Does any other system use your authentication to authenticate and authorize?
13. Do you use SSO for authentication and authorization?
14. What testability layer do I have that I can make use of to support and identify the tests?
• Does this testability layer help me to identify more tests and also classify them?

I can keep generating the questioning like this.  But I will have to pause and start working on what the questions offer me.  

With the help of these questions, I can learn better about the system before attempting to identify the test and frame it.  This also pulls out the risk or problem area if any that looks important and of priority.

I have eased my work to an extent when I know:

• the target surface area to start my work 
• what it takes and brings back, and how

In this context, I would have started this way!

The Mindset to Embrace and Encourage Automation in Software Testing

 

I was brainstorming about the automation practice setup. Before getting technical, I was looking at the mindset needed. Besides this, I was with a mentee Chidambara and answering the questions.

I'm sharing the gist of pointers that we discussed and which are essential for having the encouraging mindset towards getting started and value adding automation in testing.

 

• Build the culture that supports and encourages the Testing and Automation

• Communicate and Collaborate

  • With testers, developers, product owners, business teams

  • Build the relationship and trust; understand the system better; seek help when needed

  • Help the team to know the value they bring from their testing and automation

• Learn the problem; understand the problem; step up to solve the problem

  • Ask, is this a problem worth solving for the business

• Human is a key center of the engineering

  • Human is needed for developing

  • Human is needed for testing

  • Human is needed for automation

  • Human is needed for the maintenance of all above said

  • Humans cannot be removed from action saying automation takes care of all

• Automation is not a replacement for Testing

  • Automation exists to assist the Testing and not to replace

• Automation involves programming and is about programming to leverage the testing

  • The libraries as Selenium, Appium, Cypress, etc., are 30% to 35% of code in automation what we write

  • The rest of the code is about how we program to build, organize, scale, and maintain the automation

  • Practice programming to write better automation

  • Embrace Programming

  • Practice Object Oriented Programming and Design Patterns

• Automate which has to be automated

  • Automate first what has to be automated

  • Do not automate everything – yes not to automate everything!

  • What needs to be tested by humans, has to be tested by human

• Build the skills which will help you to do the Automation in Testing

  • This is very important and not to ignore it

• Know and confirm what is expected out of automation from business and stakeholders

  • This helps to think and define the value return from the automation

• Know what to automate, where to automate, and how to automate

  • Automating 100% is good to hear but it is not practical and logical

  • Figure out what to automate i.e. what cases or ideas to automate

  • Identify the right layer where it is appropriate to automate

    • Automating at UI layer is a cost; automate minimal at UI layer

    • Automate at service layer i.e. API layer

    • Encourage the Unit Testing practice and culture in the engineering

• Automate at the lowest layer which returns the maximum value and that has a low probability of failing

• Know what should be covered from Automation in Testing

• Understand the system being tested and its internals

  • Without understanding the use cases and business expectations out of a system, writing automation might not return the expected value

• Have the automation strategy in place than just having/working on the proof of concept

• Know and understand the tech stack of the product

  • Learn the architecture of the product

  • Understand the DevOps operation of the product

• Automation is a separate project within a project that builds the product

  • The time needed for thinking, setup, configuring, coding, maintaining, and collaborating within teams are unique tasks

  • This needs its time and it cannot be adjusted with the time available for Testing

    • Note that, Testing and Automation are two different activities though automation exists to assist the testing

  • Testing and Automation are two different activities that overlap

• Have the deterministic testability and automatability attributes in the product to leverage the value of automation and Testing

  • Ask for it with product owners and developers

• Be informed about the value returned from the automation and its limitation

  • Keep the stakeholders informed about the same

• Testability is the foundation in a system for Testing and Automation in Testing

  • Automatability attribute together with Testability makes the system very responsive for testing and automation

• Do not be scared of automation; encourage the team to start thinking about it and practice it

  • Listen to all groups who are for automation and who are not for automation

  • Understand what they mean for what is being said by them

  • Learn the context in which they are talking

  • Pick what works for you, but do not ignore and be away from the practice of automation

• Automation is a need today and for coming days; it is a skill looked for and it helps a Test Engineer very much

• Embrace automation!