Before Identifying and Listing My Tests

 

I read the below query in TTC's Telegram chat. The discussion had started on this thread and fellow members here were responding.  Further, I read this line and it made me look into it -- "The question was we have to use valid username and password..and perform a negative testcase".

The Default Thinking and Applying Interface

Including me, I see it is subconsciously common for us to approach the problem statement visualization in terms of Graphical User Interface.  When I ask why it is so, maybe it is rooted in our subconscious thinking i.e. with first order and second order or any orders of thinking.

I want to give a try to attempt approaching it by reminding and asking self the below questions:

1. Is it a GUI specific problem?
2. Is it a problem that is tied to the context of GUI?
3. What does this question encapsulate within and open as an interface?
4. What forms do these interfaces take when I stand out of specific interface?
5. Should I stick to one interface to learn and attempt this problem?

Identify the Tests and Framing of Tests

We test to learn

• Does the system do what it is supposed to do and how, why, and when?
• When the system does not do what it is supposed to do and how, why, and when?

Should I call it Negative Tests?  This is not what I share in this post.

To me, these are tests that help me to learn when the system responds and behaves in the other way than I expected.

I can start to identify the straight use cases for inputting an error (a human introduced error) at a given state/data/event; then look for the behavior of the system.  It is good when we can keep identifying and ideating the use cases.  

We get limited with use cases as we continue to think about use cases.  That said, for sure we will identify and frame the tests within identified use cases.  But, we need tests that help to learn when the system fails in doing what is supposed to do.

To supplement it there is another way, which I use.  I do not say this is the only way to supplement.  I use multiple approaches to supplement and identify the tests.  When I do so, I ask the question to the system with the help of these tests and evaluate the response of the system.

Questions to Identify the Priority Tests

I learn and understand the system each time, to identify the better tests.  And, each time I learn something new about the system that I did not know.  

When I'm asked a question in the interview, I ask for details that help me to test better or to demonstrate my deliverable better.  I will watch the questions that I ask!

If I were the candidate who got this question in an interview, I would ask the below questions.  When I learn this is good enough for the initial tests, I will pause with questions.  I move to identify and frame the tests using the responses I got for the questions that I asked.  

These questions will surely help me to be precise and close to the context that better demonstrates my testing skills.  If it is not close, then there is a problem (or a difference) in my presenting and expectations in the interview.  I will have to address it with the help of the interviewer.

Questions:

1. What is the interface where I'm entering the username and password?
• Where is this authentication used?
• On UI (if so which UI), or CLI, or touch interface, or what is its interface type?
  
• At which layer of the system this authentication is used?
2. Where is the format of username and password?
3. What is used as Authorization identity on successful authentication?
• What happens if my authentication is not successful in the UI you want me to test?
• How do I understand that UI is communicating to me that my authentication is not successful?
4. How is this authentication processed?
5. Where the authentication is mapped to authorization and stored for references?
6. What protocol is used to communicate in authentication?
• What protocol and communication order is used to grant and revoke authorization?
7. Who uses this authentication and authorization?
• To know the different means of doing the same
8. Is there any other form of authentication that grants me the authorization?
• Do these different entry points of authentication update my authorization?
• Will I have different authorization data to authenticate? If yes, how the data, states, and events are maintained for my authentication and account?
9. What's the language and Unicode supported by this system?
• Will the languages and Unicode used in the system have any impact when I try to authorize by changing the language and Unicode?  How does the system understand these differences and maintain one state of data with authorization?
10. Are there any computing differences for authentication and authorization on big and small endian machines?  If yes, how and for what context of the system's behavior, processing, and decision?
11. Where and how the authentication and authorization details are processed, stored, and presented back.
• Is there any specific reason for doing it in this particular way?
• How you have strengthened the authentication process to grant the authorization?
• For example, 1FA, 2FA, nFA, what else?
12. Does any other system use your authentication to authenticate and authorize?
13. Do you use SSO for authentication and authorization?
14. What testability layer do I have that I can make use of to support and identify the tests?
• Does this testability layer help me to identify more tests and also classify them?

I can keep generating the questioning like this.  But I will have to pause and start working on what the questions offer me.  

With the help of these questions, I can learn better about the system before attempting to identify the test and frame it.  This also pulls out the risk or problem area if any that looks important and of priority.

I have eased my work to an extent when I know:

• the target surface area to start my work 
• what it takes and brings back, and how

In this context, I would have started this way!

The Mindset to Embrace and Encourage Automation in Software Testing

 

I was brainstorming about the automation practice setup. Before getting technical, I was looking at the mindset needed. Besides this, I was with a mentee Chidambara and answering the questions.

I'm sharing the gist of pointers that we discussed and which are essential for having the encouraging mindset towards getting started and value adding automation in testing.

 

• Build the culture that supports and encourages the Testing and Automation

• Communicate and Collaborate

  • With testers, developers, product owners, business teams

  • Build the relationship and trust; understand the system better; seek help when needed

  • Help the team to know the value they bring from their testing and automation

• Learn the problem; understand the problem; step up to solve the problem

  • Ask, is this a problem worth solving for the business

• Human is a key center of the engineering

  • Human is needed for developing

  • Human is needed for testing

  • Human is needed for automation

  • Human is needed for the maintenance of all above said

  • Humans cannot be removed from action saying automation takes care of all

• Automation is not a replacement for Testing

  • Automation exists to assist the Testing and not to replace

• Automation involves programming and is about programming to leverage the testing

  • The libraries as Selenium, Appium, Cypress, etc., are 30% to 35% of code in automation what we write

  • The rest of the code is about how we program to build, organize, scale, and maintain the automation

  • Practice programming to write better automation

  • Embrace Programming

  • Practice Object Oriented Programming and Design Patterns

• Automate which has to be automated

  • Automate first what has to be automated

  • Do not automate everything – yes not to automate everything!

  • What needs to be tested by humans, has to be tested by human

• Build the skills which will help you to do the Automation in Testing

  • This is very important and not to ignore it

• Know and confirm what is expected out of automation from business and stakeholders

  • This helps to think and define the value return from the automation

• Know what to automate, where to automate, and how to automate

  • Automating 100% is good to hear but it is not practical and logical

  • Figure out what to automate i.e. what cases or ideas to automate

  • Identify the right layer where it is appropriate to automate

    • Automating at UI layer is a cost; automate minimal at UI layer

    • Automate at service layer i.e. API layer

    • Encourage the Unit Testing practice and culture in the engineering

• Automate at the lowest layer which returns the maximum value and that has a low probability of failing

• Know what should be covered from Automation in Testing

• Understand the system being tested and its internals

  • Without understanding the use cases and business expectations out of a system, writing automation might not return the expected value

• Have the automation strategy in place than just having/working on the proof of concept

• Know and understand the tech stack of the product

  • Learn the architecture of the product

  • Understand the DevOps operation of the product

• Automation is a separate project within a project that builds the product

  • The time needed for thinking, setup, configuring, coding, maintaining, and collaborating within teams are unique tasks

  • This needs its time and it cannot be adjusted with the time available for Testing

    • Note that, Testing and Automation are two different activities though automation exists to assist the testing

  • Testing and Automation are two different activities that overlap

• Have the deterministic testability and automatability attributes in the product to leverage the value of automation and Testing

  • Ask for it with product owners and developers

• Be informed about the value returned from the automation and its limitation

  • Keep the stakeholders informed about the same

• Testability is the foundation in a system for Testing and Automation in Testing

  • Automatability attribute together with Testability makes the system very responsive for testing and automation

• Do not be scared of automation; encourage the team to start thinking about it and practice it

  • Listen to all groups who are for automation and who are not for automation

  • Understand what they mean for what is being said by them

  • Learn the context in which they are talking

  • Pick what works for you, but do not ignore and be away from the practice of automation

• Automation is a need today and for coming days; it is a skill looked for and it helps a Test Engineer very much

• Embrace automation!

Time and Reflection of Time - Test Engineering

 

Time and Reflection of Time

I wish to share by writing my observations and experiences with the happenings in and around Software Testing since I started my career.  Be it technical or not, I write it so I can mark the time (past and present) of Software Testing.  The time which I experienced and experiencing!  Reflection!

I will be sharing the observations and learning of me under the label Time.  My reflection is always limited; probably constrained too, as I cannot observe the happenings and development in Test Engineering at multiple dimensions in a given time.  I will try to reflect on my learnings and understanding of the changes, adaptation, and evolvement in Software Test Engineering.

Does that happening is good or bad that is not the subject of analysis.  But, the subject is to see what it is with respect to Software Testing and Test Engineering; how it contributes to problem solving in Software Engineering.

I read and understand that time is multi-dimensional.  So is the Software Testing.  An engineer can observe, look and test a system in the multi-dimension.  But is that a need?  The time in which we are testing helps to see it with a stakeholder(s) or business's decision.

I Can Test - Debugging an Inconsistent Behavior

 

Before I Say, I Can Test

In this post, I'm demonstrating how I approached my testing and debugged an inconsistent behavior that was reported in the Telegram space of The Test Chat.  A contest that is about to start and is hosted by The Test Chat.  The title of the contest is - So You Think You Can Test?  The registration is open.  How to register detail is shared on the Telegram chat and other social media space of TTC.

The QR Code is shared in Telegram; asked to scan and submit the registration.  Here is what I observe reading through messages in the Telegram:

1. Few could scan the QR Code and could register
2. A couple of members could not see the registration form as the scanning of the QR Code failed  
3. Requests made to share the URL of the registration form than sharing the QR Code
4. Requests made to share both -- the QR Code and URL of the registration form
5. The reason -- why the QR Code is shared and not the URL
6. And the URL of the registration form is also shared now

What made me curious is, a member had replied that on multiple attempts to scan the QR Code using a mobile app, it did not fetch the URL.  This member observed the same behavior on the web, that is, on uploading the QR Code image, it did not fetch the URL.

I see a behavior now to Test Investigate and debug to learn what's happening.  With that, I have an opportunity now to say I Can Test right here on the registration procedure of So You Think You Can Test? contest.

What I did and What I Observe

It is a QR Code!  

• This QR Code shared from TTC:
• Is not like other regular QR Code I usually see
• I see black background and data with yellow foreground
• I see the Finding Pattern i.e. concentric squares in an oval shape
• These two observations are prominent in this QR Code
• I installed a QR Code reader app on my phone and scanned the QR Code
• It fetches me the registration form URL; I can open the registration form
• Then I get the question -- why those testers are experiencing a problem?
• I read through their messages and observe for clues what they have left for me
• I read the words -- drag and drop
• Ah! The web browser is as well used
• This is a very useful clue to me
• I have no idea on what desktop browser, QR Code reader websites, mobile apps, and smartphones used by these testers
• I proceed now to use an online QR Code reader
• I pick these two:
• qrreader dot online
• helloacm dot com/tools/qrcode-reader/
• I uploaded the QR Code shared by TTC on these two websites
• I see the same message and in the same format on these two websites
• The message reads -- error decoding QR Code
• Per me, this is a key observation
• What makes these two web pages show the same message and in the same format?
• I analyze the network when uploading the image and for the response I receive
• In the request
• I see the Data URL used
• Protocol mentions data
• No remote address, that is no server IP to which the request is to be sent
• Another critical observation
• I see the request initiator
• The data (jpeg image) is sent in base64 format - a binary format
• I can see the preview of the QR Code
• I see the request method as GET
• This is interesting!
• Why GET and not POST?
• I see the HTTP Status Code 200
• I see just the User-Agent in the Request Headers
• In the Response
• In the Response tab, I see the message -- This request has no response data available
• I see Content-Type: image/jpeg in the Response Headers

As I see no remote address for this request, I turned off the network.  I uploaded the QR Code image; I see the data URL fired.  Further, I observe this request is exactly similar to the one I see with the network.  So I learn, 

1. Fetching the URL from the QR Code is being done within the browser
2. I have to just launch these pages and use the QR Code images to fetch the data out of it
• No need for internet here
• And moreover, there is no remote address at all; then why the internet is needed to upload images!
3. This tells me, could be JavaScript is doing the job here!
• It is a key learning from my so far observations

Now, I look at the Console to see if I can find more hints to my test investigation.

Diving into Console and JavaScript

In the Console, 

1. For qareader dot online, I see:
• Couldn't find enough finder patterns (found 0)
2. For helloacm dot com/tools/qrcode-reader/, I see
• Couldn't find enough finder patterns

Pic: Message on qrreader dot com

Pic: Message on helloacm dot com

This is the source of the problem -- the QR Code could not be decoded.  If the Finder Patterns are not identifiable in the QR Code, then data in the QR Code cannot be decoded.  I see "found 0" in the console log.

But, why the Finder Patterns are not identified in this case though it is seen in an image by the human eyes?  This is the start of the actual test investigation and debugging for the behavior experienced.

Further, I learn both these websites make use of the same JavaScript -- llqrcode.js.  Another key learning!  I see this JavaScript is copyrighted to Lazar Laszlo.  And, I found another website that scans the QR Code image -- webqr dot com.  I experience the same behavior on uploading the TTC QR Code here as well, that is the message -- error decoding QR Code.  The same text and in the same format!

Pic: Message on webqr dot com

Reading through the below JavaScript, I make few more observations.

Pic: llqrcode.js

I learn:

1. When the image is about to be decoded
• It is taken as a 2D image
• The height and width of the QR Code image are collected and calculated
• The check is made if they are appropriate to consume and process further
2. In the process function,
• The QR Code is converted to a grayscale image
• The grayscale is the usual one that we see around us in black-and-white
3. Now trying to look for Finding Patterns,
• Looks like it is executing the condition if (h < 3)
• So the message in console -- Couldn't find enough finder patterns
• As a result, the decoding of QR is returning message -- error decoding QR Code

This information tells me, there is something to do with the QR Code shared by the TTC.  But, how come it works on smartphone apps?  

I have not attempted to fetch the code from the smartphone app and analyze it at this point in time of testing.  I make an assumption -- could be the program used in the smartphone app can identify the Finder Patterns irrespective of the color and shape in the QR Code.

I had made an observation documented in the beginning -- the Finder Pattern is in oval shape and not in the concentric square shape.

Testing the Tests

I picked the QR Code image shared by the TTC and converted it to a grayscale image.  I used this grayscale image in the above said three websites.  I see the same message -- error decoding QR Code.

I picked the registration form URL that I had got by scanning QR Code using the mobile app.  I generated the QR Code and uploaded it to these three websites.  I see the QR Code decoded successfully now; I see the Google Form URL to register.  Note that, if I turn off the internet and use a valid QR Code, I see the URL.

This tells me, there is no problem with Google Form URL accessibility or encoding or decoding.  It is to do with the QR Code shared by TTC.

Pic: QR Code shared by TTC

Pic: Grayscale QR Code generated by me

Pic: QR Code generated by me with Google Form URL

Pic: QR Code on qrreader dot online on uploading TTC QR Code

Pic: QR Code on webqr dot com on uploading TTC QR Code

The dimension and size of the QR Code image file shared by TTC are not the same as the one generated by JavaScript.  I see pixelation and a bit of distortion in the QR Code generated by JavaScript using TTC QR Code.  

Look at the Pattern Finders in the QR Code from JavaScript; compare it with the QR Code shared by the TTC.  They don't look the same.

Understanding the QR Code

QR stands for Quick Response.  QR Code was pioneered by Masahiro Hara at Japanese company Denso-Wave in 1990.  QR code has different sections and Finding Patterns is one.

I find information on these web pages useful:

• https://www.explainthatstuff.com/how-data-matrix-codes-work.html
• http://qrcode.meetheed.com/question14.php
• http://www.keepautomation.com/tips/qr_code/functions_of_qr_code_function_patterns.html.

So, What's the Problem Here?

From the inferences I'm making from my tests so far, it looks like

1. The QR Code from TTC has data (shape and color) that cannot be processed by this JavaScript?
• Not very sure!
  
• But the so far analysis says yes with the code read
2. Need to generate more customized QR Code
• If possible include Finding Patterns in different geometric shapes -- primary suspect
• Then rule out or point in if that's the problem source
• If this is not the problem, then
• The dimensions of the QR Code image file is the problem?
• For now, this is the second suspect
• But, the read JavaScript code does not say this

Need to understand how the mobile app code can read it successfully.  Then figure out the differences in the mobile app code and the JavaScript referred here.

I'm stopping my testing for now.

I can test! I test!

Bug - The One Word and Mulitple Incorrect Alternatives!?

 

What is Your Word?

I love to meet and collaborate with testers and learn what they are doing and how they are practicing Software Testing, Automation, and solving problems.  When I collaborate, I see testers using different words for communicating what we do.  For example, one will use the word "bug" and few others will use the words "issue", "defect", etc.  What word do you use when you talk about the bug and information you find with help of your testing?

The Word and Alternatives Used

These words are common in the Software Development team when referring to a bug:

• Anamoly: which means abnormality
• Defect: a legal perspective for the unexpected or not wanted or false promise
• Error: a condition due to user input or/and program execution
• Issue: a concern
• Problem: the difference between expected and reality (actual)
• Risk: a potential problem in near time
• Bug: ?

Examples:

• The abnormal growth of bone
• The defective electronic equipment that is used where people are involved
• Error on saving a file with no name
• The team has an issue with the time available
• I have a problem in sharing this report with the customer
• If we do not publish the app today we risk losing users in this season sale
• What's bugging you these days?

If observed, to an extent all the above words have something in common.  That is -- relationship, loss, cost, experience, emotions, expectations, compared to something and evaluated, feeling, and more.

But, that does not mean I can use them casually and interchangeably.  The one thing that is not so common to all is CONTEXT.  All of these have their own context and are different.

Is it a welcoming practice to equate and assign the word "bug" to all the above words?

Bug - Annoyance to whom?

The people and practitioners have different definitions tagged to the word -- Bug.  One word which looks non-technical and asks for attention in addressing it on evaluation is -- annoyance.  I won't bug you anymore!

The bug can be related to annoyance for one who has a relationship with the product or service or company.  The bug is the outcome of my experience when I look for what I need.  The bug is also the relationship that I carry with the product or service that I use.  Since I have a relationship, I feel the experience of annoyance.

I see the bug is not a word of tester or programmer or product owner or anyone in the software development team.  We use the word "bug" to communicate and document the annoyance that a user experiences in using the service we provide.  To extend the view, I can also include the business here.  If it annoys the business in delivering, giving, and making the benefits, it is a bug.  What do you say?

If my annoyance is also the same as the annoyance of an actual user and business, it carries a weightage and importance in the team.  When I say the bug report, we talk on behalf of the product, user, company that is offering service, and the team building the software.

To Sum Up the Word

It is not termed as Anamoly Advocacy, or Defect Advocacy, or Issue Advocacy, or Problem Advocacy, or Risk Advocacy.

It is termed "Bug Advocacy"!

We are trying to avoid or prevent:

• A defective product in the market that can be sued
• A problem with loss and cost for using a product or service
• A problem to business, and
• A user to be away from the [risk of] cost and annoyance for using product or service

It is up to one as a practitioner to use the appropriate word to context.  And, it must start from we Test Engineers / SDETs.  

What word to use is your choice now!  Bug?

Blog Post Update - 23rd August 2021

ISRO declared the mission of GSLV-F10 launch could not be accomplished as intended due to the technical anomaly in Cryogenic upper stage ignition.  Refer to the below image which is a tweet from ISRO mentioning the same.

Why is the word "technical anomaly" used?  And, not a bug, issue, error, or defect?  Instead, the word "anomaly" is used?  

If the word anomaly is considered as

• irregular
• not expected
• deviation from the expected
• abnormal
• something different
• deviation from commonly accepted rules or readings, and
• more...

If closely observed,

• It is a consistent output or outcome that is observed for a time period for a set of input and factors
• Any change in this outcome or output, is treated as a peculiar case?
• Or as abnormal?
• Or as deviation from what is usually observed and recorded?

Why don't we consider this as a bug, then?

• When spoken about the bug
• In the first impression, it is the emotions and experience captured or heard, or recorded
• Necessarily the root cause analysis or initial debug is not done at the time of calling out an experience as a bug

Whereas spoken about anomoly, I learn

• A set of analysis and experiments are done and the outcome is recorded
• This outcome might have shown the consistent behavior for a set of input and factors
• Anything deviation from this outcome for the same set of input and factors is considered as an anomaly?!
• Though it is unexpected, the context i.e. pre-run tests and rational experiments do not let it be treated and term it as a bug in the communication
• Instead, the word anomaly is used in communication

The Not Well Thought Talk

 

That Day

It was in 2016 April at the MITC - Moolya Internal Testing Conference.  I was a Moolya employee then and it was a conference for Moolyans by Moolyans.  I went up to the stage and shared few thoughts of me, then.  Today, when I look at what I spoke, it looks to me as not a talk that I should be doing.

The Talk

I see, my thoughts were not mindful when I spoke that day, and I was blinded.  Yeah, blinded.  This is one of the talks which I keep as an example; so that I do not share my thoughts and talk like this.  Today when I look at me of that day, I look wrong and not sound.  I question myself, "Is that me? Ah! How bad!"

Not Thoughtful, Why?

I see it is not thoughtful for these reasons:

1. It did not reflect the awareness I have on the subject and practice
2. I sent an incorrect message with my short talk for young people who were at the conference
• I should have shared how to look at it than saying what has to be done ignoring the other
• Ignoring the one and doing other is not a beneficial approach in any means
3. I'm a person and with the role who will  influence people in the direction, the work, approach, and delivery
• How can I do this as a leader?
• But a leader will also fail and a leader encounters failures often and frequently
• Leaders deal with the failures than with success often and continue what they have to do
• I failed in my thoughts and talk, that day!

None spoke or shared anything about it.  I'm not sure why.  I myself see it is not right.  Could be they would have received in the intent what I wanted to share.  But, what did I say and how, that's not right.  Ravisuriya, himself cannot take it today and acknowledges it.

What I do today!

I take my time and think before I see an opportunity or when asked to respond.  It is not that I did not think earlier prior to responding.  That day, I did not see how it will be received and interpreted.  

Today for first, I will test if I can receive and acknowledge what I'm communicating.  If this fails, I will pause and think of a better way to share and communicate.  Before sharing, I try to make sure that I have not tampered with the intent of my words and thoughts.  Can this be done each time?  I'm making a practice that I articulate and persuade better and with the right intent.

This is one of the very essential learning I have made from MITC 2016.  Thanks, Moolya and Moolyans for giving me this reflecting and retrospective learning.

Testing and Exploratory Testing

 

A Discussion in The Test Chat

In one of the discussions in The Test Chat on Telegram, the topic was "How to test without an SRS document?"  One pointer put to help in this direction was -- "Even though there is no document, you can find the requirement if you look hard enough." 

This pointer was put as a question asking how it can be done.  Further, the discussions spoke about Testing and Exploratory Testing.  

This discussion brought up the below thoughts from one of a fellow tester:

Only exploratory Testing part can be done when no document is provided , assuming the meaning by looking hard enough may lead to  various assumption 

They are suggesting to look at software under test  hard enough it seems.

And in further points they are saying know your end users and personas which is again product owners job before creating document.

In short :they are saying look hard and create SRS/BRS.

What am I seeing in the above discussions?

There is nothing wrong here with this discussion point.

I see one's learning journey in this discussion.  The good thing is -- the space as The Test Chat is available where one can share her/his thought and discuss.  All of us will have the learning journey and so do I.  It evolves as one shares her/his thoughts like this and opens up to discussion.

If we do not open up to discussion and share our thoughts like this, that's when we are not doing it right.  We have to encourage the discussion, questioning and foster an environment where it is accepted and practiced.  Only then, we will speak our learning and work, and understand it better.

Testing and the Exploratory Testing

I'm sure most of us would have crossed and crossing:

1. The job description, talks, discussion, and tasks assignment having two words -- Testing and Exploratory Testing.
2. The instruction a,s "Do the Testing and then do the Exploratory Testing".  It could also be one of your team members and manager asking "do Exploratory Testing after the regular Testing".

Few of my fellow testers use the word Exploratory Testing and Testing.  I have no problem with it.  When I discuss and write, I use the word i.e., Testing, in my practice.

Here are my thoughts for today from the learning I'm having from my practices:

• I cannot differentiate between the words Testing and Exploratory Testing.  To me, both look similar but an additional word i.e., Exploratory.
• I learn, Dr. Cem Kaner coined the word Exploratory Testing around 1982.
• Each time I test, I explore!  I could not break and create two sessions for Testing and Exploratory Testing.
• Could be there was a need then to differentiate and communicate that testing is and as Exploratory explicitly.  So did he coin the word Exploratory Testing?  I'm not sure!  
• But this has become one of the most misused words today where it is said to do Testing and then do the Exploratory Testing.  That's a different subject altogether to discuss is what I see.
• Testing is exploratory in its nature.

James Bach shared a document having the history of Exploratory Testing definition.  I'm sharing the same here with his permission.  The credits of this document are to James Bach.

Testing with no SRS or BFS

I looked and asked for SRS and BFS documents at beginning of my career.  After a year, I had no SRS documents for the product I was supposed to test.  All I had in my hands were the architecture diagram.  The team did not have the luxury of writing SRS in the timeline it was running.  This helped me to develop a skill -- to test using what I have at my hands; I did not depend on SRS documents.  I see the requirements coming in two forms -- explicit and implicit.  

• Most times, when there are no recorded documents, the documents I make on the product and system as a whole, are used for immediate reference.  Should it be called SRS/BFS?  I do not think so; I would not call it as well.  But it is part of my testing outcome i.e., by-product -- Test Notes.
• Today the FCO i.e., Feature Coverage Outline is spoken very much.  Likewise, KYC -- Know Your Customer (user of a product and stakeholder as well) is also one of the key tasks.  The testing team does this task in modeling the users and their personas so that they derive the test data and figure out the tests and scenarios.
• I have been testing with no SRS/BFS in place.  I will have to figure out what is the expectation somehow.  Else, how will I evaluate?  I use multiple approaches for the same.
• I do not crib anymore if there is no SRS/BFS.  It is one of the contexts in Software Development.  I will have to deal with it while I test and automate.  I also make sure that I'm aligned with the stakeholders for what they want and also with the market fit of the product and the problem it solves.

At this point in Time

I'm also testing for a Machine Learning product today when I write this blog post.   This project has detailed content written on the engineering and architecture of the product.  But there is no SRS/BFS written for the product.  I had challenges for a week in knowing what is what and why it exists.  The architecture content helped me.

I continued to explore the product and system; read the engineering documents and spoke to programmers when I had questions.  This helped me to test this product.  I'm testing it for 4 months now and it does not have SRS yet, though it is an ML product being built and used by customers (technology companies) for 10+ years now.  

The new features, design changes (not the UI), and fixes are being rolled out.  At this point in time, while I'm writing this line, I get a Slack ping saying -- "Upgrading and improvising the algorithm; it will take an hour.".  And another programmer pings saying, "A change in classifying algorithm for correction bug is committed and the pod is up with changes."   I'm not sure what are these changes nor it is documented anywhere. 

I will explore, figure out and test!

The Special Characters and Context

 

On creating a new password be it on a web page or mobile app or desktop application or any interface, we encounter the phrase "special characters".  And, we might see few characters represented as special characters.  Why these characters are named "special characters", here?

The Context

When one mentions "special characters" I learn and associate a context to it.  The context defines the character is special or not.  If so, why certain characters are marked as special characters for the password being created?

The context of web and HTML is a journey and evolution.  The web and HTML that existed 20 years back are not the same today.  It has evolved and so are browsers.  So the other technology i.e. desktop applications and mobile apps.

Special Characters and Context

I learn the context will make a character into a special character.  Then what's a special character?  It is a casually used phrase for the non-alphanumeric character on the keyboard.

Few of us might debate and say -- comma, colon, plus, hyphen or minus, hash, dollar, angle brackets, etc., these are all normal characters though it is non-alphanumeric.  Did people (users of the software) had special meaning for these non-alphanumeric characters in their domain of work?

But the comma, period, semicolon, hyphen, space, dollar, hash, angle brackets, etc., all have specific contextual meanings in HTML and web, and other technologies.  Do you think so?!  

The initial web technologies were not robust as today to sanitize characters and process as we do today.  Could be, for this reason, certain characters were termed as special characters and mentioned what to use and what not to use.  I'm not sure is this the reason but this could be one of the strongest reasons.

Today, the phrase "special characters" is continued to use in all major technology organization's documentation and interfaces.  Is this incorrect?  I don't know.  It helps someone to quickly relate and let her/him decide, is what I see.

Parsing and Context

Entering a password, today we assess the strength of it. There are readily available scripts and libraries that do this job.  Not sure if it was available two decades back.  Other than the security aspect of having better entropy what else is the benefit of having special characters?

Say, the special characters are those which I don't see on the keyboard layout. Then what should I think of the angle bracket (< and >) that I use in an XSS payload on the web page and behind the web page?  Note that the same angle bracket can be used in a password too.

Personally, I feel this is one of the good topics to discuss.  It can lead to learning how we term and use the word or phrase for non-alphanumeric characters.  

I don't know if this discussion is needed or not and how much it helps people who are accustomed to the phrase "Special Characters" for certain characters.  But having one does no harm and it can light up the dark areas which are unseen.

The web and desktop projects in which I worked a decade back, it had the RegEx written in different languages and scripts written in Shell, Perl, and VBScript.  These scripts and RegEx were used behind the interface to parse and validate certain characters' presence and absence.  These characters were termed as special characters in the product and it was on par with the operating system documents for consistency.  Also, there was a unique meaning and purpose for such characters here in this context.

Since these scripts and Regular Expressions were used, the characters that take a special meaning in this context were termed as Special Characters.  To keep everyone who uses the product (engineers, support, and customer) be aware of certain characters, it was termed as Special Characters in the context of product and technology.

Should Change the phrase "Special Characters"?

I don't know!

Look at the context where it is used and what characters are classified as special characters.  Changing the phrase to another phrase or word, does it solve and ease the communication with the product's users and business?  Unfortunately, not all software products might bring this change.  Having different words/phrases in the system, add additional costs?  What are those costs?

All I understand is when certain characters are classified as special characters, I look for

1. The context in which it is classified and why
2. How it is special? 
3. What differences it makes in its presence and absence?
4. Software platform terminologies on which the product runs having such classified special characters

Not fixing nor refining nor refactoring certain existence looks better in few cases!  As a technical person knowing what it is and not, is a need and helps.

Assumptions are Essentials and Necessity

 Assumptions on Assumption

I assume that after reading this post, you will use this blog post as a reference when talking about the "assumptions" in Software Testing.  We engineers, believing or hoping that the software we build will work itself is an assumption.  An assumption that is carefully thought over and evaluated.  Do you agree with that?  

When was the last time you assumed the battery charge leftover in your cell phone is enough to make a quick call or to make a banking transaction?  How did you know that the charge remaining is enough?  Anytime that assumption was broken to you?

In another way to put it out, the software and hardware we are using is an assumption that is functioning as we expected to an extent.  That also means we are testing the sets of programmed assumptions with conditions, data, states, and events when we test the software.

Did we assume when solving a problem in Math?  Did assumption help to solve?  We use theorem, hypothesis, corollary, and set of defined assumptions for the data we take in problem solving.  But some might not agree and say we do not use assumptions.  

Assumptions, Mathematics, Engineering, and Testing

I'm not sure who all will agree and disagree on saying assumptions are part of the evolution.  If there are no assumptions, probably the evolution will cease.  I buy groceries for a month assuming, I will survive this month.  Should I call this -- assumption, hope, confidence, determination, evaluation, accuracy, etc  Actually there is a very thin line of difference between the meaning of these words.  In a way, these all look alike at the certain phases in the context.

When I started testing the Machine Learning systems, this learning started to become much cognizance to me.  In fact, the AI/ML model is an assumption!  But this assumption is evaluated on the set of data that we aware of it and know what it is to an extent.

I have to come to this understanding for now on series of evaluations in my Software Testing practice:

Testing is a science of problem learning, problem solving, and deductions.  We assume certain things, and we infer conclusions from them.

If I replace the word "Testing" with Learning, Engineering, and Mathematics, I see it suits very well to my learning that I have been making as a Software Testing Practitioner.  Should I keep these assumptions I made over certain data to improvise the solution and use it as "the solution"?  That's the decision one has to take from learning out of assumptions valuing against what is expected.

Every problem solving will start, progress, and stops (not end) on the set of known assumptions made.  If one is not aware of assumptions made and being done, then is that a problem?  Or is this just as any other assumption?

Model, Assumptions, Architecture, and Testing

Do we build any engineering product without a model?  The architecture, design representation, requirement,  and strategy documentation are few models to mention here.  Then what actually is the model?

A model is a simplified version of the observations.  The simplification is for helping to focus on what has to be focused on while knowing what isn't being focused on and why.  Isn't this an assumption?  Yes, it is an assumption that is evaluated to an extent based on some (well thought?) assumption.

If we do not make absolutely no assumption about the data, then there is no reason to prefer one model over any other.  Was there any day or instance, you came to a conclusion this one test data is enough to sample the system you are testing?  Then why do people generate different payloads for XSS attacks?  Isn't that payload test data?   That payload is a model; it works or might not work.  If worked till what extent and what did it uncover?   

That payload when built by a Test Engineer (an Engineer) wasn't it an assumption? An assumption that will help her or him to discover information and help to learn about the system in a given context?

Every test data we identify, build, use or ignore is a model -- a modeled data on thought over assumptions.

Testing, Models, and Assumption

We cannot test without an assumption.  Then we cannot build an engineering system without assumptions.  At a layer, a working system is a model and in turn which is an assumption.

Rather than saying, do not assume, saying list out what you have assumed and why so.  There is no model that is prior guaranteed it works.  So the name model.  The only to know for sure which model is best in a given context is to evaluate them all.  Is it possible to evaluate all models?  In other words, is it possible to test all the models?

Since this is not possible, in practice we make assumptions that look reasonable about the system, data, state, event, user, technology, and ourselves i.e. engineers.  On making these assumptions we evaluate a few reasonable models.

Having models in the testing and automation will help in understanding and approaching the testing in the layer appropriate to the context.

Assumptions

It is not bad.  It is needed.  We use them every day in learning, problem identifying, problem solving, and decision making.  Watch out for the coverage of the assumptions made and being made.  Testing nor coding nor debugging cannot proceed further if we do use sampled and evaluated assumptions.  Question the assumption.  Questions the assumptions you are making.

HTTP2 Migration Tests to consider on AWS's ALB

 

Is your team planning to implement HTTP2 and have services hosted on AWS using ALB?  Then you might have to consider the below tests on priority for start

By default, ALB sends requests to targets using HTTP/1.1.  Refer to the Request Version and Protocol Version at https://amzn.to/3uPaxl1for sending the requests to target using HTTP/2 or gRPC.

Here are few tests to consider on priority:

1. Do all my client interfaces have migrated to HTTP2?
• If not, having ALB on HTTP2 and few clients on HTTP/1.1 will make ALB use HTTP/1.1
2. What time it takes to respond if I'm on HTTP2?
3. What is the traffic pattern i.e. how the frames (frequency & size) delivered to the endpoint on HTT2?
4. What happens to the client if there is a delay in response?
• If serving users who have low bandwidth or intermittent networks, what should be the product experience to them?
• What if you are queuing the requests and it is by the design in the system?
5. What's happening at the endpoint?
• What happens when there is a POST and PUT from the client as multi-part and not as multi-part?
• When a bunch of requests is fired at once what's the size of each batch and frame in it?
• Does the client still adhere to the HTTP/1.1 style of dealing with requests and response, or upgraded to deal the HTTP2 way?
• How the timeout is handled on the server & responded to the client it handles?
6. Do the libraries I use at the client and server end support HTTP2?

Bug Report: Applying the Single Responsibility Principle (SRP) and KISS

 

On completing my test sessions, I started to write bug reports into the tracker. I had this thought coming up again in me: "Should I keep all these problems under one bug report or have a separate one for each?".

• When we have the test with SRP (Single Responsibility Principle) and KISS (Keep it Simple and Straight), why not the bug reports?
• What's wrong if each symptom (consequence problems due to the root problem) has an individual bug report but linked to the root problem report?  

At times, I'm said to include all symptoms in one bug report along with the root cause.

• I have witnessed the symptoms of a bug do not get fixed if it is mentioned collectively in one bug report.
• Also, the linked bug reports (i.e. symptoms of the root cause) do not get fixed when the root is fixed. It will be marked as Resolved and Fixed as the root is fixed and resolved.

Hardly I have seen the symptoms as well fixed along with the root cause.

That leaves with the questions:

1. Just one bug report or separate bug reports?
2. When a test has to be specific with individual responsibility and objective, why not the bug report?
3. If the root cause is fixed does it resolve the symptoms?
• And, if the symptoms are resolved does it also resolve the root cause?

I look up to my consciousness should it be the separate bug reports or just in one bug report.  I see, I can apply the SRP and KISS to the bug report effectively.

Looking at the number of bugs is not a wise strategy. But looking at the number of problems that one root problem opens and tracing them, is useful in the engineering of a product.

Unsaid Skill embedded in the Code and Tests - Questioning

 

I read this question from Balaji Santhanagopalan in the Telegram group of The Test Chat.  This question is also on Linkedin, here.

A question to my fellow testers. How you are sharpening your questioning skills?

I see the first is to know the need for it.  And the ability to identify the question first.  In this blog post, I'm sharing how I cross through it.

I'm attempting to open my thoughts on questioning in different facets that I see:

1. Philosophical
2. Problem Learning and Identification
3. Binary and Questions
4. Pausing and Stopping with Questioning
5. Persuasion and Communication

I understand the word "skill" as:

• how do I make use of or apply what I practice, what I have or know in learning and solving a problem

Philosophical or Meta Layer Thoughts

1. When did we not use a question?
2. To better the questioning skills, we will have to identify the questions first.
3. How do we identify the questions?

How do we identify the questions?

• When I communicate
• When I communicate by showing I belong there and also do not belong there, I identify the questions. 
• And eventually, I will be identified in it

Decisions and Questions

• Everything and anything we do is by making a decision
• To decide, did we make use of the question(s)?  
• We keep making thousands of decisions every day, and 
• Each decision will have at least one question to it

For example, the last time you unlocked and locked your mobile screen, you had a question.  But that question went unnoticed.  It was at a subconscious level; it was not noticed by you consciously.  

At times what helped your subconsciousness could be a contextual question, and sometimes it can be a context-free question.  The combination of both is also possible.

Problem Identification, Learning, Understanding, and Solving Thoughts

While I use the questioning to learn, I will have to cross these challenges:

1. Keeping the questions in conscious order so that I can classify them
2. Classifying them to record it better in a contextual and context-free section
3. Will the questions annoy the one at receiving end and how to persuade so I receive a response that helps me?
4. Keeping the questions aligned with the objective yet we have context-free ones

Using the persuading skills with question and questioning will help much.  Consistently develop self-confidence and courage.

Technical Layers - Binary and Questioning

Binary - true or false.  You see that is a decision.  When I have a decision, I have a question.  The binary execution uses the conditions, states, events, data, and controls at the granular level with all interactions we have with it.

For example,

1. What happens when you unlock your screen with the fingerprint?
2. When you sign in to your email, what happens?
3. Here, internally software executes the set of conditions and the assertions by switching the controls, state, data, and events.

What are contextual questions and context-free questions in this case?  When you have a condition and an assertion in execution, you see a question?

In other words,

• the execution is an outcome of a question be it in binary or in a human

Internally the software works by asking questions and deciding what to do.  Now, what should my tests do?  Should it ask questions?

We engineers, we work with questions day in and out.  And, we do not know it consciously. 

Pausing and Stopping with Questioning

Have you come across the question

When the testing will be complete?

  

Can the testing be complete?

We would have paused the testing. Or, we would have stopped the testing.  But we never complete or completed the testing.  Testing can go and go on.  But it has to find a stop.  So is the questioning.  The questioning has to find a stop.  It can't go on and on.

When to stop questioning?  Ask when you want to stop or pause the testing.  It is important to know when to stop it.

Questions (tests) keep coming up and they may not be heard by the interfaces or people.  Learning to deal with it is a needed skill.

Art of Persuasion

When I communicate, how I persuade plays a role.  It also applies to questions I ask.

• Questioning is also communication
• Practicing the skill of persuasion will help as I continue to question

How to persuade using a question to a binary and human?  I work on this skill to assist my communication and questioning skills.

My Experiences and I

The below is missed to see on the floor by people who interact with the engineers and by the engineers.

1. A programmer has questions in the code via conditions, states, events, data, and controls.
• The product responds to do what it is supposed to do with help of these questions.
• Do you see any contextual and context-free questions here?
2. A tester has questions in the form of the tests.
• The question of testers will be to the
• code, 
• product, 
• data, 
• programmer, 
• infrastructure, 
• system, and 
• anyone who adds value to the product
• Do you see any contextual and context-free questions here?

Identifying the questions (code and tests) and their layers is one of activity I do in my practice.  I try to understand and identify myself here so we communicate.  

For example, I have to test if the host serves requests coming from the client which is not on TLS v1.2.  What are the layers and their intermediates here?  What questions do I have in my tests here?  When I look into these layers, I see the questions (code) logically?  To question the logic what is my test (question).  Here is the start!

Further, to hone or better the questioning skills, I will identify the questions in the below mentioned:

• system,
• design
• code,
• product, 
• business, 
• technology, 
• people, 
• self, and
• what you are not aware
• what you know

I will classify the questions into contextual and context-free.  

I persuade my communication and questions by making use of Test Strategy, Test Design, and Testing Techniques to drive me towards value adding questions in the context of testing. 

I will work to develop my skills, self-confidence, and courage to test, communicate, and question.  I will break my ego and hesitation and step up. I initiate the communication.  I will acknowledge the feedback. I review and refine my questions.

To start, I will start first and the rest follows.