45 Seconds of Confusion: When a Familiar GUI Fails the Human Eye

 

In meetings, we often hear the same line, 

"That's not a bug.  Report it as an enhancement."

Sometimes the observed behavior never even makes it to the enhancement list.  

But, what happens when the problem is not about functionality, but about how the users experience the GUI and its usability?

My peer Dhanasekar Subramaniam (DS) recently published a blog about a UI design that delayed him using the app. This made me curious.  How could a UI that an engineering team acknowledged and used slow down a user?

I decided to test investigate the design.  On testing and analyzing the UI behavior and usability, I discovered something interesting -- the GUI looks as in code, but behaved differently to the human eye.  

I went through the usability and experience problem, but I was conscious about this behavior so I could identify it in quick time.

If you are a SDET or Test Engineer, this blog will help you develop new perspectives while analyzing GUI problems.  If you are a manager or decision-maker, it highlights why seemingly small GUI problems should not be ignored.

When a Simple Task Creates Anxiety

Late at night 11 PM, the user opened the cab booking app Rapido, just as they had done many times before.  The goal was simple -- book a cab and reach the bus stop.

But something unexpected happened.

This time, the users could not figure out how to book the cab.

Seconds started passing.  The GUI looked familiar, yet the action to book the ride was not obvious.  Nearly 45+ seconds went by, trying to understand what to do next.

The situation made it worse.

It was 11 PM, the bus departure time was getting closer, and the user was unaware how to proceed because of the app's GUI.

That moment -- when the user knows the app, knows the task, yet cannot complete it, creates anxiety.

So the question is,

Why did two tech-savvy users, using an iPhone and familiar with the app, spend more than 45 seconds trying to figure out how to book a cab?

45 seconds for a task that usually takes less than 5 makes the problem feel bigger instantly.

Understanding the Cause of Anxiety

Here is how I started to learn and understand,

1. I installed the Rapido on the Android phone.
2. I have no ride history with Rapido.
3. I signed in for the first time.
4. I see the Ride screen.
• I see the three addresses given which I had not chosen. I could save these as favorite; I did not.

On the Ride screen, I do not see where to enter or select the pick up location and destination.  

TL;DR -- In quick and short here is what caused the confusion which led to anxiety.

1. The text in the search text field.
2. The color contrast of search text field.
3. The color contrast of view showing the three addresses.
4. The color contrast in between the 2 and 3.
5. User not able to identify that is a search text field which is tappable.

What to fix?

1. To rephrase the search text.
• "Enter pickup location" works as charm; refer Pic-4 in this blog post.
2. To have better color contrasting for the three GUI elements.
• The GUI color and contrast to have ΔE ≥ 3  -- good and preferred
3. To highlight a search text field so that I will be prompted to tap and enter or select the pick-up and drop location.
4. When experimenting with AB Test configs, the GUI design to follow the suggested GUI Design & Color Engineering practices.

In usability and user experience what is not noticed is as good as not present.

Continue to read the below sections for the detailed information on usability and user experience problems.  

If you want to quickly know the technical analysis alone then move to sub-section -- Why It Fails - Mathematical Analysis and Human Brain. This tells why the present Rapido app's GUI Design and colors confuses a human brain and eye.  

No wonder, why the users got into anxiety when booking the cab at 11 PM!

The First Usability Pitfall in the GUI

Now continue to read with much attention.

1. I looked at the top of the screen.  I see text -- "Where are you going?".
2. Below the text, I see the three locations listed which I had not chosen or entered or of my current interest.

Ah! that confused me.  Why?

I closely looked my mobile screen, that is Ride screen.

Pic-1:  The confusing text and three locations displayed

1. I see a search text field.
2. I see a search icon next to the text field.
3. The search text field has a text -- "Where are you going?"

This is the first usability pitfall in the confusing GUI.

Why am I asked where am I going and listing the three locations that I did not enter or choose?

The Second Usability Pitfall in the GUI

In the below image Pic-2, I see,

1. There is no prominent visual difference in the layouts of
• Search text field
• Color hex code #FFF8FAFC
• The three location displayed
• Color hex code #FFF6FAFF
2. Yes my brain could not perceive the differences right away between these two layouts.

Pic-2: The color contrast of the GUI elements.

The color contrast of these two layouts are almost same.

• This added for the confusion.
• My brain was perplexed in knowing what's happening here.
• I'm wasting time here to learn how to book a cab.
• Is this what a Rapido need as a business?
• Or, does it need a user to book a cab right away on opening the app?
• Won't this experience drive away the user to the competitors app -- Ola, Uber, Namma Yatri.

If my brain cannot perceive the differences and is processing to understand what's happening here, is that a good user experience?

Forget about the user experience.  Is that a serving UI Design and Engineering?  I will leave this to your thoughts.

Further, the space in between these two layouts is with color contrast of #FFFFFFFF.  This makes the confusion much stronger.  Why?  

• All these three GUI components are on one main view
• To the human eye and brain, the color contrast of these three GUI components blend as one rather three distinct GUI elements.

This is the second usability pitfall in the confusing GUI.

Not being able to distinguish between these three GUI elements in a quick time is a problem.  Why the app has confusing color engineering for these three GUI components?  Why the GUI design did not highlight that search text field as tappable?  Why the search text is confusing when combined with the GUI color?

If the GUI components had distinguishing contrast colors.

Rapido's Competitor GUI and Usability

The competitor of Rapido has similar GUI, but it is more intuitive with the search text and color contrast of GUI components.  Refer the below pic.

In the Ola and Uber apps,

• The search text is straight and easy to understand.
• The readability of search text is close to the context of using the app.
• Importantly, the search text field is easily distinguishable easily.

The search text and distinguished search text field makes me understand I should tap on it and enter the pick up and destination location.

Pic-3: The search text field and GUI in Ola, Uber and Rapido apps.

The Two GUIs of Ride Screen

When test investigating, I experienced the two GUIs of Ride screen.  

The other GUI looks better in terms of usability and prompted me to tap on the search text field.  But my question is when do I get this screen?

Could be a AB test parameter coming in to the app shows the different Ride screen.  I did not pick this for debugging as it looked better.

In the below picture, screen 2 shows a better search text.  Also, it does not have those three locations that I see in screen 1.

Pic-4: The two different Ride screens of Rapido.

Test Investigation & Analysis - Why My Brain & Eyes Took 45+ Seconds?

This section has details of my debugging, test investigation and analysis.  I have put my eyes, brain, smart phone, reasoning and the Rapido app to evaluation.

If you are a Test Engineer or SDET in a role, this should be super helpful when you are testing for GUI.  Do not skip it!

Human Eyes and Cones for Blue Shades

I learn, the human eyes have three cone types L, M and S; below are it sensitivity.

• L-cones is for Long-wavelength cones; it is sensitive to red-ish light.
• M-cones is for Medium-wavelength cones; it is sensitive to Green-ish light.
• S-cones is for Short-wavelength cones; it is to Blue-ish light.

You remember, I shared the hexa color code for the two GUI components, that is

• #FFF8FAFC
• #FFF6FAFF

Pic-5:  #FFF8FAFC color

Pic-6:  #FFF6FAFF color

In between the above two hexa color code images, we have white (#FFFFFFFF) background as in the Rapido app.

These two hexa color codes explain my observation.

• I struggled to distinguish the subtle color difference, especially in certain ranges.
• So the two users who were booking the cab at 11 PM.  Why?  
• We humans have a fewer S-cones and it is less sensitive.  Hence, the small changes in blue/cyan hues are hard to see.

But, the small changes in red/green are easier to detect.  

Have you seen sky in the night when an aero plane is flying.  

You see the red-light of an airplane though the distance between sea level and the air plane is around 10 to 13 KM.  

Why do the plane use red-light and not blue or any shades of blue?  Hope this should trigger your eyes and mind now.  

With this simple daily life example shared, tell me about the two blue shades discussed here with minimal difference and next to each other as GUI components in an mobile application?  

To add more to the complexities, the hardware and display capabilities of smart phone models from the same OEM varies.  You see, how critical is the UI Engineering now in software design!

Display Behavior of Smart Phones

Even before your and my eyes see the color, the smart phone's display (hardware + software) processes it.

That is, the smart phones,

• Quantize colors (round up values)
• Use OLED sub-pixels
• Apply gamma corrections

This leads the rgb(246, 250, 252) and (248, 250, 252) to be same emitted light.  Why?  The display hardware will round or merge the small difference.  An another reason why on the Android device that I used and on an iPhone the other two users could not differentiate between the two GUI elements of Rapido app.

Viewing Angle Makes It Even Worse

I was holding my smart phone at 180 degree to the ground -- that is device at an angle to view. 

Pic-7:  Holding the smart phone to view at an 180 degree to the ground.

At an angle,

• The contrast reduces
• The colors shift
• The subpixels blur

So even a small difference that might exist becomes visually flattened.  Small hue differences are flattened by the panel optics.  This effect is common in cyan/blue hues.

Further our human visual system averages nearby pixels.  The two adjacent colors like #FFF8FAFC and #FFF6FAFC are interpreted by the brain as a single averaged blue.

Why It Fails - Mathematical Analysis and Human Brain

In color science, the term "Empfindung" is used when talking about the experience of a color.  It is a German word meaning sensation or perceived differences.

In the UI and Design Engineering it is used as ΔE.  Where Δ (delta) means change or difference, and E is Empfindung -- perceptual sensation.

The professional UI Engineering rule enforces the minimum color difference of 3 to 5 RGB units.  Or the perceptual metrics as ΔE > 3 to ensure UI elements remain distinguishable. 

For these two colors #FFF8FAFC and #FFF6FAFC in discussion here, the calculation using CIEDE2000 color difference formula results in ΔE = 2.0 to 2.3.

• This range falls into interpretation as slightly noticeable -- borderline perception.

But, both colors in discussion here are with very high lightness (almost white) and low chroma (very low saturation) -- this is critical. 

• For such colors the human sensitivity to differences drops significantly.  
• Despite the ΔE ≈ 2, in reality the users will not notice the differences, especially on the mobile phones.

The smart phone display may map the two colors discussed here to the same or near-identical output.  Why?

• Display Quantization
• The values of R and B are near maximum in the above said color
• The maximum color value is 255.
• In our case the RGB of the two are rgb(246, 250, 252) and (248, 250, 252).
• Rounding and Gamma correction will compress the differences from the hardware and software of a mobile device.
• OLED Screen
• The smart phones having OLED screens, with high brightness levels,
• The subpixel differences become less distinguishable
• +3 in blue channel may not produce a visible shift
• -2 in red may be completely lost
• Viewing Conditions
• On smart phones, the brightness varies, ambient light interferes and viewing angle shifts color.
• As a Result, ΔE ≈ 2 is often perceived as identical by human brain.
• That is, the human brain cannot differentiate between the colors

Using these two colors #FFF8FAFC and #FFF6FAFC for buttons, states and backgrounds is risky and leads to users unable to distinguish them reliably.

For those with accessibility concerns and conditions, ΔE ≈ 2 is effectively invisible.  It fails practical usability and experience expectations.

The final outcome from the test investigation and debugging is,

1. These two colors used is not helpful and unreliable.
2. Not suitable for distinguishing the GUI elements.
3. Needs stronger contrast for interactive GUI design.
4. If the ΔE ≈ 2.0 to 2.3, it is borderline and unreliable.
• The range 2.0 to 2.3 may be ok only for subtle background variations.
• In this case it failed; we all three users had difficulty and trouble to understand the GUI.
5. Use the colors and contrasts with ΔE ≥ 3.

Use the below as a reference (heuristic) for standard perception thresholds.

• ΔE is < 1,
• the interpretation is not visible.
• ΔE is 1 to 2, 
• the interpretation is barely noticeable.
• ΔE is 2 to 3, 
• the interpretation is slightly noticeable
• But it does not serve on mobile app engineering.
• ΔE > 3, 
• clearly visible

On the lighter fun side refer to the below pic.  Let me know what is the Empfindung of your eyes for the discussed two colors together with the white background.

Pic-8:  The screenshot of this blog post on my mobile screen.

The three colors FFFFFFFF, FFF8FAFC and FFF8FAFC appear to merge and look as one color.  Doesn't it?  

You can  try an experiment with the above pic.  Look at this pic, by increasing and decreasing the brightness and contrast of the screen (smart phone or monitor) by being in the lighted and dark.  What's your experience?

Hope this should be a sufficient data to understand the seriousness of the problem discussed in this post.

What's the Fix?

1. For mobile app engineering, the recommendation for GUI color and contrast is
• ΔE ≥ 3  -- good and preferred
• ΔE ≥ 5  -- safe
2. Use the better text in the search text field
• This looks better and prompts to tap on it -- "Enter pickup location"
3. Distinguish and highlight the search text field GUI component prominently
4. When experimenting with AB Test configs, the GUI design to follow the suggested GUI Design & Color Engineering practices.

These fixes also benefit the users with accessibility concerns and conditions.  

Any questions or information needed on this please do connect with me.  I'm just one ping away!

Performance Testing - The Unusual Ignorance in Practice & Culture

 

I'm continuing to share my experiences and learning for100 Days of Skilled Testing series.  I want to keep it short and as a mini blog posts.  If you see, the detailed insights and conversations needed, let us get in touch.

The ninth question from season two of  100 Days of Skilled Testing is

What are some common mistakes you see people making while doing performance testing?  How do they avoid it?

Mistakes or Ignorance?

It is mistake when I do an action though I'm aware that it is not right in the context.

I do not want to label what I share in this blog post as mistake.  But, I call it as ignorance despite having or not having the awareness, and the experience.

The ignorance said here is not just tied to the SDLC.  It is also tied to the organization's practice and culture that can create problems.

To this blog post's context, I categorize the ignorance in these categories -- Practitioner and Organization.

1. Practitioner's ignorance
• Not understanding the performance, performance engineering, and performance testing
• When said performance testing, taking it as - "It is load testing"
• No awareness on what is performance and performance engineering
• Going to the tools immediately to solve the problem while not knowing what is the performance problem statement
• Be it web, API, mobile or anything,
• Going to one tool or tools and running tests
• No much thinking on how to design the tests in the performance testing being done
• Ignoring Math and Statistics, and its importance in Performance analysis
• No idea on the system's architecture, and how it works
• Why it is the way it is?
• The idea of end-to-end is extended and used in testing for performance and having hard time to understand and interpret the performance data
• How many end-to-end your tests have identified?
• Can we test for performance to all these identified and unidentified end-to-end?
• Relying on the resource/content in internet and applying or using it in one's context without understanding it
• No idea on the tech stack and how to utilize the testability offered by it in evaluating the performance
• Not using or asking for testability
• Getting hung to most spoken and discussed 2 or 3 tools on the internet
• Applying tools and calling out it as performance testing
• No attempting to understand the infrastructure and resources
• How it impacts and influences the performance evaluation and its data
• Idea on Saturation of resources
• Thinking it as a problem
• Thinking it as not a problem
• Not working to identify where will be the next bottleneck when solving a current bottleneck
• What to measure?
• How to measure?
• When to measure?
• What to look when measuring?
• Not understanding the OS, Hardware resources, Tech Stacks, Libraries, Frameworks, Programming Language, CPU & Cores, Network, Orchestration, and more
• Not knowing the tool and what it offers
• I learn the tool everyday; today, it is not the same to me compared to yesterday
• I discover something new that I was not aware of what it offered and exist
• I learn the new ways of using the tool in different approaches
• No story in the report with information/image that is self-describable to most who reads it
• And, more; but the above said resonates with most of us
2. Organization's ignorance
• At the org level, for first and to start, it is ignorance in Performance Engineering
• Ignoring the practice of performance engineering in what is built and deployed
• Thinking and advocating, increasing the hardware resources will increase and better the performance
• In fact, it will deteriorate over a period of time no matter how much the resources are scaled up and added
• Ignoring the performance evaluation and its presence in CI-CD pipeline
• The performance tests on CI-CD pipeline should not take beyond few minutes
• What is that "few minutes"?
• Not prioritizing the importance of having the requirements for Performance Engineering

Recently, I was asked a question - How to evaluate the login performance of a mobile app using a tool "x"?

In another case, I see, a controller having all HTTP requests made when using web browser. Running these requests and trying to learn the numbers using a tool.

I do not say this is wrong way of doing.  That is a start.

But, we should NOT stay here thinking this is a performance engineering and that is how to run tests for learning a performance aspect[s].

To end, the performance is not just - how [why, when, what, where] fast or slow?  If that is your definition, you are not wrong!  That is a start and good for start; but, do not stick on to it alone and call performance.   It is capability.  It is about getting what I want in the way I have been promised and I expect; this is contextual, subjective and relative.  The capability leads to an experience.  What is that experience experienced?

Sometimes, serving the requests by what you call as slow, is a performance.  What is slow, here?

The words fast and slow are subjective, contextual and relative.  It is one small part of performance engineering.

That said, let me know, what have you been ignoring and unaware in practice of Performance Engineering & Testing?

Deep Link and its Testing via Automation

 

I get these question consistently from my fellow testers and community.

1. How to automate the mobile apps and web applications using Deep Links?
2. How to automate the business flows using Deep Links?
3. How to achieve end-to-end business flows testing on using Deep Links?
4. How to automate scenarios in mobile apps using Deep Links?
5. What is the best approach to automate the mobile apps using Deep Links?
6. What is the best practice to automate using the Deep Links?

And, more questions on same pitch.

No Deep Dive into - What is Deep Link?

A hyperlink in HTML is a kind of deep link within a website or to another website.

Deep Link is known with different names for web, Android app and iOS app.  All these names have the same understanding and intent at some point.

The Deep Links are URIs that takes me directly to a specific part (activity or fragment) of the app that I'm using or testing.  The Deep Link will have an intent which tells where I will be taken on using it.

When we converse on diving deep technically into testing and automation of Deep Link, will share more insights into its internals.

Deep Link and Challenges

This question is discussed with me often:

How to do end-to-end testing using the Deep Link?

Automation of a mobile app using Deep Link poses a challenge which is not experienced in web application.  

One such challenge is, say you have not installed the mobile app.  [This is solvable!]

• On using a Deep Link, I should be taken to Apple Store or Play Store based on the app.
• I have to install the app.
• Post this, in the traditional automation, I should start traversing the business work flows via GUI.
• Is this adding to the flakiness aspect of automation via GUI?

When we talk so much about flakiness and how to avoid (not prevent), should we exercise business workflows when automating using Deep Link?  What you are thinking?  Let me know!

Scoping of Automation Using Deep Link

Back to the fundamentals.

• We have to automate, no escape from it.  Let us automate what must be automated!
• Let us not fall into trap of "Automate everything!"
• For today, I'm in this mindset and attitude,
• What we automate depends on the objective or goal that we want to accomplish.
• Each test should have precise and deterministic goal.
• A test via automation is not an exemption to it.
• A test defined in automation should be precise, deterministic and have a single objective - Single Responsibility Principle.

What is the objective of my testing via automation for the Deep Link?  This define the scope and extent of my automation.  This will minimize the number of checks that I do using Deep Link.

The purpose of Deep Link is to take me to specific part of the mobile app.

• Should I start the end-to-end or exercising the workflow to be included in the Deep Link tests?
• If included, am I not complicating the testing via automation?

Automation using Deep Link

I ask this question to myself and to my team.

What is the goal of testing via automation using Deep Link?

This question helps me to pick minimal and necessity flow actions.   It has lead and leads me to define minimal tests for Deep Link based on what we want to learn from automation of same.

To me, the purpose of Deep Link is not end-to-end testing.  It's purpose is,

Am I taken to the intended state and data when used the Deep Link?

I have kept the test intent to this.

With this, I have come with tests that has minimal must evaluation and assertion to learn if the app is responding or not to the Deep Link.  This is what the business wants when the Deep Links are created.

The app usage and workflow function is not a problem statement of Deep Link in a general context.

Deep Link is not for end-to-end.  It is to take to you from a point to another point, that's it.

Are you automating using Deep Link?

App Crash! Testing around and inside the crash

In day and out, I come across testers, programmers, managers, and management having efforts on fixing all the crashes. Yes, all the crashes.  In a way I see, if the app did not crash, I will not know the areas that is not being handled well enough.  My testing focus areas will also have tasks noted in such areas to test and learn as much as possible. I do that task provided I can make/given time for it as it is unplanned task.

The common checks to handle crash!?

I learn, an exception if unhandled at runtime, it leads to crash.  There are multiple exception that app can witness which we never thought of during development.  In my initial days of testing, I was in assumption, if we can have -- null checks, index check, illegal argument check, and state check for an activity, we have handled most of the exceptions.  I learned, I'm wrong! How many checks can I write in the code.  I'm not a programmer by job.  I'm a tester. 

I see these checks are not enough and few more got added to my test strategies eventually -- race conditions, unexpected data, wrong data, environment factors, and many more. The collection of these checks is continuing to grow.  Do I cover all these possible crash inducer collections in my testing?  No, I can't and I won't have that luxury of time as well.  Technically, I will learn and prioritize what to use and when.

How the check looks in code, to me?

I write code for automation, which I need to assist my testing.  Here I did write such checks.  At a point of time, I saw, the automation code was full of checks.  Is that a right way?  Definitely not!  A professional and skilled programmer might not do that.  If a programmer has to have such check in each layers of the app architecture, will that sound good?  Personally as a tester, I will not design my tests that way.  As I'm not a programmer, I'm not aware of the pros and cons of doing so. At least I know that is not a better practice to have checks in each layer of architecture of the app.

By handling the exception in my automation code, I print stacktrace of exception.  But will I learn from it to be a better tester?  That's the question I have asked and continue to ask myself.  The exception fix I'm doing, is it stopping me from learning the problem which I have introduced by the actions I'm doing on the app? Is that exception blocking me from learning the underlying problem in my automation code and app?  If yes, then I have a fundamental problem which I have to work upon is my thought.

Why it crashes?

Why at all the app gets into crash? I learn, if the app gets into a state it was not designed for, it will and should crash. As a tester, I will have to learn this state (and such states) at the earliest when I experience crash and on reading the crash stacktrace. I will be happy and not make a fuss about it, if I see a crash for first.  

I learn what is the priority and impact of the crash?  Should I invest my time and test investigate further to provide as much as details to programmer?  Or should I report it with good enough details and continue my testing?  I will answer this question to me.  All I wish is, I don't want the user of the app to experience the crash.  If there is a crash, as a member from development team my intent is to keep it to minimal number having little or no impact.  I see the crash is a great source of learning about my work in the app.

I used to be fussy about crash years back be it on desktop applications, database, web applications and mobile apps.  Now, I have come to point, I love them and it is absolutely okay for an app to crash and it should crash.  What I do post crash in fixing, that tells the bigger story.  In my work, the crashes have made the app better because the team was serious about those crashes.

What on having the crash?

Should the user lose the data what she or he entered on experiencing the crash?  I personally, don't want this to happen for me.  If it happens, I will be annoyed!  That said, how to handle it?  That's something we will have to sit with the programmers and team, for discussion.  

At what point in app, encountering the crash, should we close the app and start over again. At what point in app, it is okay to note the crash and pull the stacktrace, and let continue the user using the app with data entered intact. At what point in app, I should not show UI in view on entering crash and what to show, then resume over safely from there on.  Personally I feel this is a team effort and not just a programmer's effort in making it happen.  

Few testing strategies hack to uncover crash

Here are few things I do and I ask my fellow testers to do when testing mobile apps:

1. Using the test data which will check the data integrity in app at -- entry point, during processing and post processing.
2. Identifying the states of the app and passing the invalid states in app at -- entry point, during processing and post processing.
3. Identify the input which is not from a tester and user. Classify the input on which I don't have control. For example -- the incoming intent; the app responding to APIs (default values, entered values and processed values); the app receiving the response from APIs; the device state; app's activity life cycle state and data/state exchange, and many more as this.
4. Depending on Android or iOS app, much more strategies can be narrowed down to be specific and work.  At the end, what time I'm left with in the test cycle and what do business want, directs me on what to do.

Debugging and Investigation skills

There are libraries which collects the crash on exception along with other details as -- device info, user info and network info.  I have been with programmers and had difficulty in reproducing the crash and experience it in development environment.  This said, are the logs enough to fix crash?  May be, so we handle the exception and continue the flow of app in runtime.  But did we solve the root problem that caused the crash?  No!  This is where, I feel, the skill of a tester comes in and it is very much needed.

This skill defines me what I'm as a tester and value I bring to the system.

Native, Web and Hybrid - How related and different these apps are?

During discussion on mobile apps with a tester, one of the question was - "What is native app? What is the difference between hybrid app and native app?" Like one uses the mobile apps in and out almost all days on mobile devices, yet there is confusion in knowing what these apps are and the differences among them.  In this post, I do not share about advantages and disadvantages for the types of mobile apps, rather I share what I understand for what they are.  And, this should help to start learning on advantages and disadvantages if any.

Further knowing what is the technology stack being used for the native mobile app, web app and hybrid mobile app, a tester can design the tests better.  This helps to figure out the testability and from here one can get the idea which are good to automate and assess, and those which requires human skills to assess further and test.

Mobile web app and mobile device

I will walk through briefly on native mobile app and hybrid mobile app in this post. Before I do that, let me tell bit about web app with Responsive Web Design (RWD) for mobile devices.  Do you know any website that you use often? Open that site on your mobile device browser now and experience how it is rendered. Now, why not changing the orientation of your device and view it in portrait and landscape. What was your experience? Were you able to use it on mobile device? Did it render in same width and height as in desktop browser or it gut customized to your mobile device's screen?

Along with RWD, HTML5 and latest web technologies of JS with CSS, their exists frameworks to build web app for mobile which uses the nativity of mobile devices. They are evolving and exists, today.  One such example is Ionic framework.

While mobile OS extends the support of web view to display the web contents within the activity of an app, the above said web technologies and framework supports to build mobile web app.  With this the mobile web app can make use of device's hardware and sensor, as a native mobile app does.

This mobile web app is not available as an app in the Google's Play Store and Apple's App Store. However it can look like a mobile app in an experience but the implementation way differs. When a website can be rendered to mobile device's screen size, is it possible now to get a feel like not a browser based website though address bar is shown?  Frameworks available today to build the mobile web app will help to do so.

Some examples for the mobile web app are GMail, Flipkart, Amazon, and Facebook.



Native app and mobile device

Mobile app which is installed on mobile device and that can make use of hardware and sensor available on mobile device, is typically a native app.  This app is built for a particular mobile device's OS and platform.  With mobile device's few apps come pre-installed and user can still look for an app what she or he needs on Google's Play Store and Apple's Play Store.

A native app can access what it requires on mobile device provided it has the permission to do so. Now make a list of permissions you have seen which app asks you for installing and using it on the mobile device.

Some example for the native mobile app are Contacts, Message, Camera, Calendar, and the one which can be downloaded from the respective store of Google and Apple.

Note: In this context, I'm taking Google and Apple platforms as it is well known to most.



Hybrid app and mobile device

In above said, HTML5 and JS technologies have the major role in mobile web app. Now, the same can be used to build apps which can be installed on Android an iOS?

Technically the hybrid app will have HTML5 web app within a native wrapper which is supported by the mobile OS.  This mobile app is developed with HTML5, JS technologies and CSS, then compiled into native Android or native iOS or any other mobile OS using wrapper support.

Apache Cordova is one such mobile application development framework which  helps to wrap up an app that can run in WebView by having nativity extension to mobile device's (OS) platform. The plugins of Cordova can be used to access the device's hardware support. On knowing the word as Cordova, now if happened to explore further we will hear the words -- Phonegap, Titanium, Ionic, Xamarin etc. Knowing how they support to develop mobile application will help to understand better about the hybrid mobile app.

If the hybrid mobile app is also a kind of web app, then the HTML, CSS and JS files are stored in the app which will be installed on the device? Or, will it get from the server on launch of the web on a desktop browser?  This depends on how the hybrid app will be designed. If it is packaged into the .apk or .ipa file, then web APIs can be used send and received data.  If it is received from server then a layer around the WebView will have to built and web APIs will be still used.

The apps built in this sort are called hybrid because

• they are neither native apps as the rendering is on via WebViews instead of platform's UI framework
• they are neither web apps because they are packaged as an app for distribution across platforms using the platform nativity via its APIs

Have you used, GMail app on your phone by downloading it from store? Are you using Instagram, Facebook, Youtube, Maps apps on your mobile device on downloading it from store? Then that is an example of the hybrid app which combines the strength of native app and mobile web app together.

So whats the point

To end this post, if asked what is best i.e. whether native app or hybrid app, it depends on context and what is trying to be solved with negotiation.  Both helps to accomplish the needs of app developing organization, programmer, tester and user.  While each have they their own strength and weakness, it is the choice to be made by people who builds for whom.  In this post, I have not shared the advantages and disadvantages of it.

Now starts the fun of exploration around testing and automation.  How to test the apps built in hybrid way? What are the level of isolation I can identify so that I can layer it out layer by layer and test? Where to start if it is getting packaged into nativity of the platform on mobile OS?

If you are doing it or figured it out, pull me to pair with you and test together. Let us do it!

Android: Test investigating for RAM consumption - Bit more on Garbage Collection

From this post on knowing about the heap, we moved to this post which gave brief thought on the GC. Continuing the understanding of heap, I will share what I have gathered for garbage collection's algorithms.  It is upto to choice of people to select and implement algorithm(s) and technique(s) in the JVM they build.  To remind, Android OS has Dalvik (till 4.x) and ART (from 5.x) JVMs. For understanding and carrying out the test investigation for RAM consumption, being aware of the GC algorithm and techniques is a plus point.

Roots and Garbage Collection

The GC algorithm will be doing these two things.

1. It must find the dead objects and live objects
2. It must reclaim the heap space used by the dead objects and make it available for application/program

Finding the dead objects is accomplished by identifying set of root and enumerating it to determine the reachability from the roots. An object is reachable if there is a path of references from the roots by which the program/application can access the object. The reachable objects are called 'live' objects and those which are not reachable is called 'dead' object i.e. the garbage.  Any objects referred by a live objects is also reachable and it is 'live' object.

What may be the GC algorithm chosen in the JVM, the JVM will stop the running application when it has to run the GC. When this happens, every thread except for the threads needed for the GC will stop their task for a while. The interrupted threads will resume only after the GC is completed.

So there is a need for fine tuning the algorithm and techniques of GC used in the JVM, such that the interrupted time will be as small as possible. Dalvik and ART are no exception from this.

Source of the "Roots" in JVM

The executing program/application will have access to root, always. The root set in a JVM will be based on the implementation. Few of the source of roots can be as below

• object references in local variables
• operand stack of any stack frame
• object references in any class variables
• strings stored in the heap (from loaded classes) -- for example, class name, method name, field name

Approach to identify live objects among dead objects

Reference counting and Tracing are the two common approach for identifying the live objects (reachable) from the dead objects (not reachable) in the heap.

The reference counting GC differentiates the live objects from the dead objects by keeping a count for each object in the heap.  This count keep the track of numbers of references to that object. The tracing GC will trace out the graph of references starting from the root nodes.  The objects that are encountered during the trace, will be marked and it is known be live object. The one which are not marked in the tracing are the dead objects and needed to removed from the heap.

Further learning and exploring

It will be not in scope to write more about the different algorithms and techniques with this blog post. However, use the below to learn more about the GC algorithm, types and techniques. It will help in knowing how the ART better.

1. Generational Collectors -- Weak generational hypothesis; Young generation; Old generation; Permanent generation; 
2. Serial GC
3. Parallel GC
4. Parallel Compacting GC
5. Concurrent Mark and Sweep GC
6. Garbage First GC
7. Reference counting collectors
8. Tracking collectors
9. Compacting collectors
10. Copying Collectors
11. Adaptive Collectors
12. Remembered sets and popular objects
13. Reference objects -- reference object and its referent
14. Reachability state changes and use of Cache, Canonicalizing mapping, and Pre-Mortem clean up

Android: Test investigating for RAM consumption - Garbage Collection

Getting the brief information on heap this post, I will move on to Garbage Collection in JVM. I have tried to understand Garbage Collection and here is the brief information on the same.

Garbage Collection

When an object is no longer referenced by a program, the heap space it occupies it can be recycled so the space is made available for subsequent new objects.

The JVM has to to keep tracking which objects are being referenced by the program which is executed, and finalize and free unreferenced objects at run time.  This activity will need more CPU time than what it requires when program explicitly free unnecessarily occupied memory.  How to optimize the CPU usage by the program is key in designing of the product so product can still do better when CPU is loaded in a given context.

Heap Fragmentation and Garbage Collection

The heap fragmentation occurs gradually as the program execution happens. The new objects are allocated and the unreferenced objects are freed. The freed portions of heap memory are left in between portions occupied by live objects. In this case, the request to allocate new objects will be filled by extending the size of the heap even though there is sufficient unused total space in the existing heap.  And this is possible if there is sufficient contiguous free heap space available into which the newly created object can fit.

The swapping is required to service the growing heap and this can degrade the performance of the program which is in execution.  Now, I will leave it to you to consider what and how will the impact be from heap fragmentation in desktop and mobile device, running on lower RAM. Have you ever noticed message saying "Out Of Memory" or "running out of memory"?

With the garbage collected dead objects, the problem is, it can impact the performance of the application/program. The JVM has to keep track of the objects being referenced by the executing program/application and finalize the unreferenced objects (dead objects) in run time. All this happens on the fly while one uses the program/application.  This activity can consume the CPU scheduling time. What if the program/application itself freed the memory? Won't it consume the CPU, then? Both consumes but how much is the question. It depends on the code of the application/program and the hardware as well. I'm not sure if one can handle the CPU scheduling in garbage collection process to free the dead objects. If it is possible to handle the CPU scheduling in this context, programmers should have the constraints and limitations; else by this time most of the app would have been better in GC environment if the CPU scheduling was in full control to the written code of program/app.



Graphical Representation of GC and Heap Fragmentation

Figure-1: Garbage collection contextual elements

Figure-2: Heap Fragmentation from garbage collection

Android: Test investigating for RAM consumption - Heap and Heap Dump

The process of allocating new objects and removing the unused objects to make space for the new allocation is termed as Memory Management.  

Object Memory Management - Heap and Stack

Hearing the two terms Stack and Heap, can bring confusion in the memory management. Though I will not get in detail about each here, I will share briefly for what I have understood.  

The 'stack' memory is used for static memory allocation and 'heap' is used for dynamic allocation. For those allocated on the 'stack', they are stored directly to the memory and the allocation happens when the program is compiled. Whereas for those allocated on 'heap' have their memory allocated at the run time and the size of heap is limited by the virtual memory. Both the 'heap' and 'stack' are stored in RAM.

Shallow Heap and Retained Heap

Moving ahead, in 'heap' we will hear the words -- Shallow Heap and Retained Heap.

• Shallow heap -- is the amount memory consumed by one object. That is, the amount of memory allocated to store the object itself by not taking the reference objects into consideration. The shallow heap size of set of objects is the sum of shallow sizes all objects in the set.
• Retained heap -- is the amount memory consumed by all objects that are still alive by objects at the root of the retained set. The retained heap size of an object is, "shallow heap size" added with "shallow heap size of objects that are accessible directly or indirectly, only from this object".
• Retained set -- is the set of objects which will be removed by garbage collection when an object or multiple objects is garbage collected.

In other way, the shallow heap of an object is the size of object in the heap. The retained size of the same object is the amount of heap memory that is freed when this object is garbage collected.

From these two resources (1 and 2), I have understood the details of Shallow Heap and Retained Heap. I feel it is simple and well explained; I have been referring it since years.

Heap dump

It is a snapshot of the memory at a Java process at a given point of time. The dump will consist of Java objects and classes. The reference of memory for the objects and classes will be available in the dump taken. If the Garbage Collection occurs at time of obtaining the dump, then the obtained heap dump will have details and information about the remaining (existing & used, existing & unused) objects and classes.

The heap dump do not tell who created the object and where it was created. Based on type of heap dump, the information available in the dump will be as below.

1. Classes -- Classloader, name, super class, static fields
2. Objects -- Class, primitive value and references, fields
3. Thread Stacks and variables -- The information about call stack of threads at time of obtaining heap dump. The information about local variables and objects.
4. GC Roots -- The mentioning of objects which are reachable by JVM

Benefits of sharing the debug APK with tester

On knowing -- the myths around debug APK; what is debugging? and how tester can debug and be of help to programmer?, I tell about the benefits.  The programmer, product owner and team will benefit by sharing the debug APK to tester for testing. However, make sure you are in synchronous and mutual agreement with the tester if there is a 'concern' in sharing so.

"benefits" to programmer in sharing Debug APK to tester

Test investigating using log

With the debug APK the code written to print system messages, debug messages, and stack trace will print the information into the log.  While testing, tester can learn the logs to monitor -- what's happening within the device; what's happening to the app and what's happening in the environment where the device is. Won't this help to debug and investigate the problem or incident? The additional information shared by the tester apart from the incident won't it help programmer to the trigger or cause of the incident?

Test investigating with variables and breakpoints

The source code of app is available to tester and tester is using the debug app on her/his device or emulator. During this time tester can watch the variables, execution sequence and flows, knowing the stack details around the break points and analyze the code. This will help the tester to be stronger and get skilled each time. As a return, it comes back in testing to your product/app. Getting to frame level debugging and testing, wow! Is not that a value add?  If the tester need assistance to start here, giving few hours initially is a worth investment!

Tracking and testing for consumption and allocation

Yes, testing for allocations and use of HEAP. Further testing for network consumption, battery consumption, CPU consumption and UI performance. Is this not a value add in building high performance mobile app?

"limitation" with benefits in context of tester

As the benefit, there are also limitations in what a tester can do in context.  One such context is tester not having the source code of the app but has the debug APK.

Here, the tester might not be able to look into variables, stack values and test using the breakpoints. But, the tester can decompile the APK and work with compiled code with breakpoints and watch the execution flows. This is time consuming task and the task which can end abruptly if insertion of breakpoint and flow is paused incorrectly. Reporting the problem out of this observation is not suggested unless it is very sure that there is a problem.  But this not stop from testing for consumption and allocation by the app on the device.

As this, there can arise several context which yields the limitation. Figuring out how better the testability can be visualized and making use of it in testing, is the skill of tester.  Everyone in the team will have role to help tester to acquire and hone such skills.

"why" the delay in giving a debug APK to your tester?

If you are a programmer and product owner, then on knowing these benefits, yet if you are delaying to share a debug APK with tester, then there is a concern which needs to be addressed by project management.  This is what I learn.

If you are a tester and have come to know the value you can add from your testing, it will be a unfair to be quite and not asking for the debug APK.

"closing" thought with iOS and WP debug app

As with Android APK, the iOS and WP apps also have debug apps.  The benefit of it is quite similar. As well, there are additional benefits when testing in the respective programming environment with it's devices with the debug app.

Explore the benefits. Encourage the better and useful testing for the context of product.

Debugging and the debug APK

As a textbook or reference book reader, debugging is "the process of identifying and fixing of problem that stopped the product to work as intended."

"debugging" - as how I see

As a practicing tester, I see, "debugging is a empirical activity which involves investigation, reasoning and analysis on the buggy/risky/enhancement incident to learn the details about it, so that a fix solution will be found for it." 

There are different techniques in debugging and the person doing the debugging may carry out multiple activities in parallel.  Discussing the different techniques, approaches and activities in debugging is not with scope of this writing. Hence I'm not sharing much about it, in this post.

I learn, "having the debugging attribute in the product is testability strengthening factor."

"no!" to debug APK while USB Debugging is enabled?

In my testing practice, I test for mobile apps as well. I test Android, iOS and WP apps.  In the context of this blog post writing, I will stick on to Android app and Android APK.

It is very common for a programmer to have "USB Debugging" option enabled on her/his Android device by turning on 'Developer' permission. So the programmer will be able to install the Android app while it is being developed. Further the programmer will use this installed app for testing and debugging, typically.

When tester enable 'Developer' permission and choose "USB Debugging" option to assist in debugging the problems in app, won't that help in developing app? Won't it reduce the time and efforts of the programmer in analyzing the problem when the tester write the detailed Test Investigation report from the debugging? To a tester, who aspire to be skilled mobile app tester, it is very usual to have this setup on her/his test or personal device as the programmer will have.

If so, then have you ever said "no!" to share the debug app to your tester who has same interest in the product as the programmer and product owner?  Why? I will leave it to you for introspection.

"few" myths around sharing the debug APK

There are myths which gives fear in mind when asked for the debug APK. I have tired to demystify few of them here and help to be confident in sharing the debug APK with your tester on mutual understanding and agreement in common interest.

"types" of debugging in developing Android APK

I understand, the IDE used to program the Android APK will use the Java debug by default. That means, the code compiled in debug mode will generate the APK which works with tools based out of Java to see the internals of app.

Along with this, if the IDE supports different configurations of debugging, it can be used. I have not used apart from 'Java' type so far. Hence I cannot share anything further on this.

By the way, when I say "developing", I mean programming along with testing.


"differences" between the debug APK and non-debug APK

When said non-debug APK, the assumption is that it will be a release APK. The debug APK is for the testing purpose. In practice, the release APK will have debug attribute to false. Beside this, the certificates used to sign the APK will be different when it is debug APK and the release APK. These certificates will be bound to date for expiry.

Both debug and non-debug APKs can be ZIP aligned and signed.  Making sure the log writing code is available in the app is essential to print system messages, debug messages and stack trace.

next Post

In the next blog post, I will share the benefits what the programmer and interested stakeholder can get by sharing the debug APK with the tester.

Few myths around sharing the debug APK

The debug APK is a useful build which helps to learn the internal aspects of the app. If security is a concern with the debug APK, then it has to be handled in building the security layers for the app and it's code. Saying the debug APK is the gateway to security problems is not thoughtful, technically and logically.

Here are few myths and fear that is observed when asked for the Android debug APK.

Myth-1: Given the release build/app it cannot be turned to debug build/app

One can use the release build/app and turn it to the debug build though the programmer do not share one. However, there is no assurance that this converted debug build will be functional as the release build/app. If tester converts it to debug build and reports the behavior which are not valid then it is additional effort to programmer and project to triage and then say it is not valid. This happens!

Myth-2: One will have access to the code of debug app

One will have access to code when one installs the app on her/his device. The question is in what format and how the access will be available. Apps having the obfuscation with help of ProGaurd and DexGaurd will obfuscate the compiled code of app. To deobfuscate it, one will require the mapping file key. Further one can go and ahead and work to deobfuscate with different approaches and techniques with no mapping file available as well.  It is upto what level the security is strengthened for app.  Anyways the code is available (in a format) when one installs the release APK/app. 

Myth-3: One can tamper the debug app and use it illegally

From above said myth's opinion, I will continue for this topic.  While building the release APK, the programmer will sign the APK with certificate. The release build certificate and debug build certificate are not same. The person who wants to use tamper the app, still she/he can decompile, edit the app, deobfuscate, recompile, sign the app and build it. When signing on recompile, if the person do not have valid certificate still she/he can go ahead with self created test certificate.  But, the question is will this certificate work well for the app which has multiple legacy SDKs included and expecting a certificate and key. In my test practice, I have seen, the app failing to function for its purpose though it launches in such context.

Myth-4: One can misuse and learn the sensitive area and information from debug app

With out the actual source code, one can reach to stage where she/he can see the register and stack values by using just the debug app's compiled code. For getting into this level, one has to successfully decompile and build it back to SMALI data structure and then start the debugging to look into the stack registers and flow sequence. If the obfuscation is applied at this level, then it a round more of strengthening. Yet it does not mean one cannot cross it but the time needed is unknown.  This is possible with non-debug app i.e. release APK as well.

Myth-5: One can know what happens in the debug app while using it

The very purpose of the debug app is this i.e. to know whats happening when using the app. The question is -- who is using the debug app? Without the actual source code of the app, the information learning will become narrow as the testing progresses with debug app.  

Myth-6: Giving the debug is like giving the permission to code of app

Debug APK means not permission to code of app for editing and building.  If there are other checks on the decompile, recompile and signing to stop app from functioning after this, the APK will not be usable as the release APK.

Myth-7: One can gain the control over the app with debug app

Unless the decompiled app is edited in code and built again to do illegal and incorrect activities, it is unfair to think as this. If this is the possibility with the Android app which is being built, then it has to be handled in how the structuring happens for security layers of the APK. This is nothing to do with debug APK.

If the debug app is pointing to the staging or test environment and access is restricted with IP white listing and similar other approaches, it will restrict the access from public use. The other way is to reduce the validity time for debug signing certificate. Keeping this validity that is reasonable for the test cycle is another approach. After this validity the debug APK will not get installed on devices unless it is decompiled and resigned again.