I Can Test - Debugging an Inconsistent Behavior

 

Before I Say, I Can Test

In this post, I'm demonstrating how I approached my testing and debugged an inconsistent behavior that was reported in the Telegram space of The Test Chat.  A contest that is about to start and is hosted by The Test Chat.  The title of the contest is - So You Think You Can Test?  The registration is open.  How to register detail is shared on the Telegram chat and other social media space of TTC.

The QR Code is shared in Telegram; asked to scan and submit the registration.  Here is what I observe reading through messages in the Telegram:

1. Few could scan the QR Code and could register
2. A couple of members could not see the registration form as the scanning of the QR Code failed  
3. Requests made to share the URL of the registration form than sharing the QR Code
4. Requests made to share both -- the QR Code and URL of the registration form
5. The reason -- why the QR Code is shared and not the URL
6. And the URL of the registration form is also shared now

What made me curious is, a member had replied that on multiple attempts to scan the QR Code using a mobile app, it did not fetch the URL.  This member observed the same behavior on the web, that is, on uploading the QR Code image, it did not fetch the URL.

I see a behavior now to Test Investigate and debug to learn what's happening.  With that, I have an opportunity now to say I Can Test right here on the registration procedure of So You Think You Can Test? contest.

What I did and What I Observe

It is a QR Code!  

• This QR Code shared from TTC:
• Is not like other regular QR Code I usually see
• I see black background and data with yellow foreground
• I see the Finding Pattern i.e. concentric squares in an oval shape
• These two observations are prominent in this QR Code
• I installed a QR Code reader app on my phone and scanned the QR Code
• It fetches me the registration form URL; I can open the registration form
• Then I get the question -- why those testers are experiencing a problem?
• I read through their messages and observe for clues what they have left for me
• I read the words -- drag and drop
• Ah! The web browser is as well used
• This is a very useful clue to me
• I have no idea on what desktop browser, QR Code reader websites, mobile apps, and smartphones used by these testers
• I proceed now to use an online QR Code reader
• I pick these two:
• qrreader dot online
• helloacm dot com/tools/qrcode-reader/
• I uploaded the QR Code shared by TTC on these two websites
• I see the same message and in the same format on these two websites
• The message reads -- error decoding QR Code
• Per me, this is a key observation
• What makes these two web pages show the same message and in the same format?
• I analyze the network when uploading the image and for the response I receive
• In the request
• I see the Data URL used
• Protocol mentions data
• No remote address, that is no server IP to which the request is to be sent
• Another critical observation
• I see the request initiator
• The data (jpeg image) is sent in base64 format - a binary format
• I can see the preview of the QR Code
• I see the request method as GET
• This is interesting!
• Why GET and not POST?
• I see the HTTP Status Code 200
• I see just the User-Agent in the Request Headers
• In the Response
• In the Response tab, I see the message -- This request has no response data available
• I see Content-Type: image/jpeg in the Response Headers

As I see no remote address for this request, I turned off the network.  I uploaded the QR Code image; I see the data URL fired.  Further, I observe this request is exactly similar to the one I see with the network.  So I learn, 

1. Fetching the URL from the QR Code is being done within the browser
2. I have to just launch these pages and use the QR Code images to fetch the data out of it
• No need for internet here
• And moreover, there is no remote address at all; then why the internet is needed to upload images!
3. This tells me, could be JavaScript is doing the job here!
• It is a key learning from my so far observations

Now, I look at the Console to see if I can find more hints to my test investigation.

Diving into Console and JavaScript

In the Console, 

1. For qareader dot online, I see:
• Couldn't find enough finder patterns (found 0)
2. For helloacm dot com/tools/qrcode-reader/, I see
• Couldn't find enough finder patterns

Pic: Message on qrreader dot com

Pic: Message on helloacm dot com

This is the source of the problem -- the QR Code could not be decoded.  If the Finder Patterns are not identifiable in the QR Code, then data in the QR Code cannot be decoded.  I see "found 0" in the console log.

But, why the Finder Patterns are not identified in this case though it is seen in an image by the human eyes?  This is the start of the actual test investigation and debugging for the behavior experienced.

Further, I learn both these websites make use of the same JavaScript -- llqrcode.js.  Another key learning!  I see this JavaScript is copyrighted to Lazar Laszlo.  And, I found another website that scans the QR Code image -- webqr dot com.  I experience the same behavior on uploading the TTC QR Code here as well, that is the message -- error decoding QR Code.  The same text and in the same format!

Pic: Message on webqr dot com

Reading through the below JavaScript, I make few more observations.

Pic: llqrcode.js

I learn:

1. When the image is about to be decoded
• It is taken as a 2D image
• The height and width of the QR Code image are collected and calculated
• The check is made if they are appropriate to consume and process further
2. In the process function,
• The QR Code is converted to a grayscale image
• The grayscale is the usual one that we see around us in black-and-white
3. Now trying to look for Finding Patterns,
• Looks like it is executing the condition if (h < 3)
• So the message in console -- Couldn't find enough finder patterns
• As a result, the decoding of QR is returning message -- error decoding QR Code

This information tells me, there is something to do with the QR Code shared by the TTC.  But, how come it works on smartphone apps?  

I have not attempted to fetch the code from the smartphone app and analyze it at this point in time of testing.  I make an assumption -- could be the program used in the smartphone app can identify the Finder Patterns irrespective of the color and shape in the QR Code.

I had made an observation documented in the beginning -- the Finder Pattern is in oval shape and not in the concentric square shape.

Testing the Tests

I picked the QR Code image shared by the TTC and converted it to a grayscale image.  I used this grayscale image in the above said three websites.  I see the same message -- error decoding QR Code.

I picked the registration form URL that I had got by scanning QR Code using the mobile app.  I generated the QR Code and uploaded it to these three websites.  I see the QR Code decoded successfully now; I see the Google Form URL to register.  Note that, if I turn off the internet and use a valid QR Code, I see the URL.

This tells me, there is no problem with Google Form URL accessibility or encoding or decoding.  It is to do with the QR Code shared by TTC.

Pic: QR Code shared by TTC

Pic: Grayscale QR Code generated by me

Pic: QR Code generated by me with Google Form URL

Pic: QR Code on qrreader dot online on uploading TTC QR Code

Pic: QR Code on webqr dot com on uploading TTC QR Code

The dimension and size of the QR Code image file shared by TTC are not the same as the one generated by JavaScript.  I see pixelation and a bit of distortion in the QR Code generated by JavaScript using TTC QR Code.  

Look at the Pattern Finders in the QR Code from JavaScript; compare it with the QR Code shared by the TTC.  They don't look the same.

Understanding the QR Code

QR stands for Quick Response.  QR Code was pioneered by Masahiro Hara at Japanese company Denso-Wave in 1990.  QR code has different sections and Finding Patterns is one.

I find information on these web pages useful:

• https://www.explainthatstuff.com/how-data-matrix-codes-work.html
• http://qrcode.meetheed.com/question14.php
• http://www.keepautomation.com/tips/qr_code/functions_of_qr_code_function_patterns.html.

So, What's the Problem Here?

From the inferences I'm making from my tests so far, it looks like

1. The QR Code from TTC has data (shape and color) that cannot be processed by this JavaScript?
• Not very sure!
  
• But the so far analysis says yes with the code read
2. Need to generate more customized QR Code
• If possible include Finding Patterns in different geometric shapes -- primary suspect
• Then rule out or point in if that's the problem source
• If this is not the problem, then
• The dimensions of the QR Code image file is the problem?
• For now, this is the second suspect
• But, the read JavaScript code does not say this

Need to understand how the mobile app code can read it successfully.  Then figure out the differences in the mobile app code and the JavaScript referred here.

I'm stopping my testing for now.

I can test! I test!

Bug - The One Word and Mulitple Incorrect Alternatives!?

 

What is Your Word?

I love to meet and collaborate with testers and learn what they are doing and how they are practicing Software Testing, Automation, and solving problems.  When I collaborate, I see testers using different words for communicating what we do.  For example, one will use the word "bug" and few others will use the words "issue", "defect", etc.  What word do you use when you talk about the bug and information you find with help of your testing?

The Word and Alternatives Used

These words are common in the Software Development team when referring to a bug:

• Anamoly: which means abnormality
• Defect: a legal perspective for the unexpected or not wanted or false promise
• Error: a condition due to user input or/and program execution
• Issue: a concern
• Problem: the difference between expected and reality (actual)
• Risk: a potential problem in near time
• Bug: ?

Examples:

• The abnormal growth of bone
• The defective electronic equipment that is used where people are involved
• Error on saving a file with no name
• The team has an issue with the time available
• I have a problem in sharing this report with the customer
• If we do not publish the app today we risk losing users in this season sale
• What's bugging you these days?

If observed, to an extent all the above words have something in common.  That is -- relationship, loss, cost, experience, emotions, expectations, compared to something and evaluated, feeling, and more.

But, that does not mean I can use them casually and interchangeably.  The one thing that is not so common to all is CONTEXT.  All of these have their own context and are different.

Is it a welcoming practice to equate and assign the word "bug" to all the above words?

Bug - Annoyance to whom?

The people and practitioners have different definitions tagged to the word -- Bug.  One word which looks non-technical and asks for attention in addressing it on evaluation is -- annoyance.  I won't bug you anymore!

The bug can be related to annoyance for one who has a relationship with the product or service or company.  The bug is the outcome of my experience when I look for what I need.  The bug is also the relationship that I carry with the product or service that I use.  Since I have a relationship, I feel the experience of annoyance.

I see the bug is not a word of tester or programmer or product owner or anyone in the software development team.  We use the word "bug" to communicate and document the annoyance that a user experiences in using the service we provide.  To extend the view, I can also include the business here.  If it annoys the business in delivering, giving, and making the benefits, it is a bug.  What do you say?

If my annoyance is also the same as the annoyance of an actual user and business, it carries a weightage and importance in the team.  When I say the bug report, we talk on behalf of the product, user, company that is offering service, and the team building the software.

To Sum Up the Word

It is not termed as Anamoly Advocacy, or Defect Advocacy, or Issue Advocacy, or Problem Advocacy, or Risk Advocacy.

It is termed "Bug Advocacy"!

We are trying to avoid or prevent:

• A defective product in the market that can be sued
• A problem with loss and cost for using a product or service
• A problem to business, and
• A user to be away from the [risk of] cost and annoyance for using product or service

It is up to one as a practitioner to use the appropriate word to context.  And, it must start from we Test Engineers / SDETs.  

What word to use is your choice now!  Bug?

Blog Post Update - 23rd August 2021

ISRO declared the mission of GSLV-F10 launch could not be accomplished as intended due to the technical anomaly in Cryogenic upper stage ignition.  Refer to the below image which is a tweet from ISRO mentioning the same.

Why is the word "technical anomaly" used?  And, not a bug, issue, error, or defect?  Instead, the word "anomaly" is used?  

If the word anomaly is considered as

• irregular
• not expected
• deviation from the expected
• abnormal
• something different
• deviation from commonly accepted rules or readings, and
• more...

If closely observed,

• It is a consistent output or outcome that is observed for a time period for a set of input and factors
• Any change in this outcome or output, is treated as a peculiar case?
• Or as abnormal?
• Or as deviation from what is usually observed and recorded?

Why don't we consider this as a bug, then?

• When spoken about the bug
• In the first impression, it is the emotions and experience captured or heard, or recorded
• Necessarily the root cause analysis or initial debug is not done at the time of calling out an experience as a bug

Whereas spoken about anomoly, I learn

• A set of analysis and experiments are done and the outcome is recorded
• This outcome might have shown the consistent behavior for a set of input and factors
• Anything deviation from this outcome for the same set of input and factors is considered as an anomaly?!
• Though it is unexpected, the context i.e. pre-run tests and rational experiments do not let it be treated and term it as a bug in the communication
• Instead, the word anomaly is used in communication

The Not Well Thought Talk

 

That Day

It was in 2016 April at the MITC - Moolya Internal Testing Conference.  I was a Moolya employee then and it was a conference for Moolyans by Moolyans.  I went up to the stage and shared few thoughts of me, then.  Today, when I look at what I spoke, it looks to me as not a talk that I should be doing.

The Talk

I see, my thoughts were not mindful when I spoke that day, and I was blinded.  Yeah, blinded.  This is one of the talks which I keep as an example; so that I do not share my thoughts and talk like this.  Today when I look at me of that day, I look wrong and not sound.  I question myself, "Is that me? Ah! How bad!"

Not Thoughtful, Why?

I see it is not thoughtful for these reasons:

1. It did not reflect the awareness I have on the subject and practice
2. I sent an incorrect message with my short talk for young people who were at the conference
• I should have shared how to look at it than saying what has to be done ignoring the other
• Ignoring the one and doing other is not a beneficial approach in any means
3. I'm a person and with the role who will  influence people in the direction, the work, approach, and delivery
• How can I do this as a leader?
• But a leader will also fail and a leader encounters failures often and frequently
• Leaders deal with the failures than with success often and continue what they have to do
• I failed in my thoughts and talk, that day!

None spoke or shared anything about it.  I'm not sure why.  I myself see it is not right.  Could be they would have received in the intent what I wanted to share.  But, what did I say and how, that's not right.  Ravisuriya, himself cannot take it today and acknowledges it.

What I do today!

I take my time and think before I see an opportunity or when asked to respond.  It is not that I did not think earlier prior to responding.  That day, I did not see how it will be received and interpreted.  

Today for first, I will test if I can receive and acknowledge what I'm communicating.  If this fails, I will pause and think of a better way to share and communicate.  Before sharing, I try to make sure that I have not tampered with the intent of my words and thoughts.  Can this be done each time?  I'm making a practice that I articulate and persuade better and with the right intent.

This is one of the very essential learning I have made from MITC 2016.  Thanks, Moolya and Moolyans for giving me this reflecting and retrospective learning.

Testing and Exploratory Testing

 

A Discussion in The Test Chat

In one of the discussions in The Test Chat on Telegram, the topic was "How to test without an SRS document?"  One pointer put to help in this direction was -- "Even though there is no document, you can find the requirement if you look hard enough." 

This pointer was put as a question asking how it can be done.  Further, the discussions spoke about Testing and Exploratory Testing.  

This discussion brought up the below thoughts from one of a fellow tester:

Only exploratory Testing part can be done when no document is provided , assuming the meaning by looking hard enough may lead to  various assumption 

They are suggesting to look at software under test  hard enough it seems.

And in further points they are saying know your end users and personas which is again product owners job before creating document.

In short :they are saying look hard and create SRS/BRS.

What am I seeing in the above discussions?

There is nothing wrong here with this discussion point.

I see one's learning journey in this discussion.  The good thing is -- the space as The Test Chat is available where one can share her/his thought and discuss.  All of us will have the learning journey and so do I.  It evolves as one shares her/his thoughts like this and opens up to discussion.

If we do not open up to discussion and share our thoughts like this, that's when we are not doing it right.  We have to encourage the discussion, questioning and foster an environment where it is accepted and practiced.  Only then, we will speak our learning and work, and understand it better.

Testing and the Exploratory Testing

I'm sure most of us would have crossed and crossing:

1. The job description, talks, discussion, and tasks assignment having two words -- Testing and Exploratory Testing.
2. The instruction a,s "Do the Testing and then do the Exploratory Testing".  It could also be one of your team members and manager asking "do Exploratory Testing after the regular Testing".

Few of my fellow testers use the word Exploratory Testing and Testing.  I have no problem with it.  When I discuss and write, I use the word i.e., Testing, in my practice.

Here are my thoughts for today from the learning I'm having from my practices:

• I cannot differentiate between the words Testing and Exploratory Testing.  To me, both look similar but an additional word i.e., Exploratory.
• I learn, Dr. Cem Kaner coined the word Exploratory Testing around 1982.
• Each time I test, I explore!  I could not break and create two sessions for Testing and Exploratory Testing.
• Could be there was a need then to differentiate and communicate that testing is and as Exploratory explicitly.  So did he coin the word Exploratory Testing?  I'm not sure!  
• But this has become one of the most misused words today where it is said to do Testing and then do the Exploratory Testing.  That's a different subject altogether to discuss is what I see.
• Testing is exploratory in its nature.

James Bach shared a document having the history of Exploratory Testing definition.  I'm sharing the same here with his permission.  The credits of this document are to James Bach.

Testing with no SRS or BFS

I looked and asked for SRS and BFS documents at beginning of my career.  After a year, I had no SRS documents for the product I was supposed to test.  All I had in my hands were the architecture diagram.  The team did not have the luxury of writing SRS in the timeline it was running.  This helped me to develop a skill -- to test using what I have at my hands; I did not depend on SRS documents.  I see the requirements coming in two forms -- explicit and implicit.  

• Most times, when there are no recorded documents, the documents I make on the product and system as a whole, are used for immediate reference.  Should it be called SRS/BFS?  I do not think so; I would not call it as well.  But it is part of my testing outcome i.e., by-product -- Test Notes.
• Today the FCO i.e., Feature Coverage Outline is spoken very much.  Likewise, KYC -- Know Your Customer (user of a product and stakeholder as well) is also one of the key tasks.  The testing team does this task in modeling the users and their personas so that they derive the test data and figure out the tests and scenarios.
• I have been testing with no SRS/BFS in place.  I will have to figure out what is the expectation somehow.  Else, how will I evaluate?  I use multiple approaches for the same.
• I do not crib anymore if there is no SRS/BFS.  It is one of the contexts in Software Development.  I will have to deal with it while I test and automate.  I also make sure that I'm aligned with the stakeholders for what they want and also with the market fit of the product and the problem it solves.

At this point in Time

I'm also testing for a Machine Learning product today when I write this blog post.   This project has detailed content written on the engineering and architecture of the product.  But there is no SRS/BFS written for the product.  I had challenges for a week in knowing what is what and why it exists.  The architecture content helped me.

I continued to explore the product and system; read the engineering documents and spoke to programmers when I had questions.  This helped me to test this product.  I'm testing it for 4 months now and it does not have SRS yet, though it is an ML product being built and used by customers (technology companies) for 10+ years now.  

The new features, design changes (not the UI), and fixes are being rolled out.  At this point in time, while I'm writing this line, I get a Slack ping saying -- "Upgrading and improvising the algorithm; it will take an hour.".  And another programmer pings saying, "A change in classifying algorithm for correction bug is committed and the pod is up with changes."   I'm not sure what are these changes nor it is documented anywhere. 

I will explore, figure out and test!

The Special Characters and Context

 

On creating a new password be it on a web page or mobile app or desktop application or any interface, we encounter the phrase "special characters".  And, we might see few characters represented as special characters.  Why these characters are named "special characters", here?

The Context

When one mentions "special characters" I learn and associate a context to it.  The context defines the character is special or not.  If so, why certain characters are marked as special characters for the password being created?

The context of web and HTML is a journey and evolution.  The web and HTML that existed 20 years back are not the same today.  It has evolved and so are browsers.  So the other technology i.e. desktop applications and mobile apps.

Special Characters and Context

I learn the context will make a character into a special character.  Then what's a special character?  It is a casually used phrase for the non-alphanumeric character on the keyboard.

Few of us might debate and say -- comma, colon, plus, hyphen or minus, hash, dollar, angle brackets, etc., these are all normal characters though it is non-alphanumeric.  Did people (users of the software) had special meaning for these non-alphanumeric characters in their domain of work?

But the comma, period, semicolon, hyphen, space, dollar, hash, angle brackets, etc., all have specific contextual meanings in HTML and web, and other technologies.  Do you think so?!  

The initial web technologies were not robust as today to sanitize characters and process as we do today.  Could be, for this reason, certain characters were termed as special characters and mentioned what to use and what not to use.  I'm not sure is this the reason but this could be one of the strongest reasons.

Today, the phrase "special characters" is continued to use in all major technology organization's documentation and interfaces.  Is this incorrect?  I don't know.  It helps someone to quickly relate and let her/him decide, is what I see.

Parsing and Context

Entering a password, today we assess the strength of it. There are readily available scripts and libraries that do this job.  Not sure if it was available two decades back.  Other than the security aspect of having better entropy what else is the benefit of having special characters?

Say, the special characters are those which I don't see on the keyboard layout. Then what should I think of the angle bracket (< and >) that I use in an XSS payload on the web page and behind the web page?  Note that the same angle bracket can be used in a password too.

Personally, I feel this is one of the good topics to discuss.  It can lead to learning how we term and use the word or phrase for non-alphanumeric characters.  

I don't know if this discussion is needed or not and how much it helps people who are accustomed to the phrase "Special Characters" for certain characters.  But having one does no harm and it can light up the dark areas which are unseen.

The web and desktop projects in which I worked a decade back, it had the RegEx written in different languages and scripts written in Shell, Perl, and VBScript.  These scripts and RegEx were used behind the interface to parse and validate certain characters' presence and absence.  These characters were termed as special characters in the product and it was on par with the operating system documents for consistency.  Also, there was a unique meaning and purpose for such characters here in this context.

Since these scripts and Regular Expressions were used, the characters that take a special meaning in this context were termed as Special Characters.  To keep everyone who uses the product (engineers, support, and customer) be aware of certain characters, it was termed as Special Characters in the context of product and technology.

Should Change the phrase "Special Characters"?

I don't know!

Look at the context where it is used and what characters are classified as special characters.  Changing the phrase to another phrase or word, does it solve and ease the communication with the product's users and business?  Unfortunately, not all software products might bring this change.  Having different words/phrases in the system, add additional costs?  What are those costs?

All I understand is when certain characters are classified as special characters, I look for

1. The context in which it is classified and why
2. How it is special? 
3. What differences it makes in its presence and absence?
4. Software platform terminologies on which the product runs having such classified special characters

Not fixing nor refining nor refactoring certain existence looks better in few cases!  As a technical person knowing what it is and not, is a need and helps.