A day ends in seeing crash and hearing "No crash! It works."

It is a case where I involved actively in the second day to complete my investigation, after learning my negligence and assumption.  The happening, in the case, stopped the testing for a day. 

A day with no testing and programming, it is a cost which has not turned into any benefit to business! I heard the same from business team.

Happening in the case

One of my fellow tester had got a project assignment and expectation was to carry out the performance test. The latest release and deployment was available for the testing and it was used. While the fellow tester started the task from tester's desk, the business team started using the product from their place.

The tester noticed the product crashing after few actions. Clueless why it is, just the message was available that said, "Something went wrong!"  Looking at this, tester had rolled back the latest installation and did a fresh installation. Yet, the same behavior and message.  This made the tester to use different test setup and noticed the same behavior and message.

By this time, I was said about this behavior.

Observing the happening

Fellow tester walked through me the context of task, it's priority and expected time to finish the task. Following with that walk through, tester gave me the happening in the test environment and the observations recorded for the crash.

With this, the tester said, the business team is not facing this behavior. But here it is crashing after few actions on launching it.

Hearing this, we wanted to make sure the version of the product and hardware setup was similar or close enough to be fair.  It looked everything is symmetrical in terms of product version and setup.

I asked for the log and noticed the parsing failing at the client end. Running through the stack trace in the log it was evident for me that there is a problem in processing the data. Which data, that was the question.  The stack trace said, the data that was wrong. But the claim from the tester was, the same data is being used by testing team and business team. Then, why the crash experience just for the tester.

Product getting crashed is a common sight. The interesting aspect in the crash is knowing the root. I said this to tester and asked to observe data transmitted and update me on the same.

How negligent here, I'm!

I knew, from so the so far investigation, we are very close in knowing the root cause of the problem. But then, from here, on updating what to do next in investigation and analyze, I moved into my practice. This is the mistake I did. I should have joined the tester in completing it. 

But that next part of investigation did not happen for the whole day and it never came to me back. I heard business team is using product without any problem. At least, I should have inquired with the engineering team but which I did not.

Next day's fresh listening

I noticed the discussions between business and engineering team. The calculation was on the time that went out seeing no testing and no clue of the crash.  After few minutes, I heard from tester, it works if I choose different network. It was tried in morning and since it worked, the other network line is used.

That shook me very strongly and my eyes just became very straight in sight seeing the engineering team. I asked, "How? That should not be a problem at all from network information I see here. It should be something else, not the network."  

I could not convince myself that changing the network will solve the crash. Just the product did not exhibit the behavior for a reason. What was the reason? I started to converse with the product again.

Going back to the undone

I requested for the data from the data monitoring between the client and server. This was one part of the investigation which was suppose to continue.  My bad, I assumed, this was done and engineering team saw no problem here. I did not think of asking it and look at it, on the previous day, because I assumed.

This time, I was very keen to look and requested to setup environment for monitoring the data. Used different networks and noticed data sent, received and parsed.

What's the problem?

From last day's investigation it was evident to me, the client could parse the data and assign it to the object. The consequence was Null Pointer Exception and product getting crashed.

Now with data, I started analyzing and every bit of data and line by line information in the log.  The problem is, the product is unable to handle the data at this transition point if that is not from the server. But why? Isn't that a expectation? I left the question with the engineering team for their discussion.

The request from client did not reach the server at all. The web monitoring and filtering system (WMFS) in the network it did not let the request to reach server. The response was from the WMFS to client.

The other mystery to learn was, how is it possible to work on one network and not on another network line, while the service line is same.  In one network, out of the first three request, all got response from WMFS.  While on another network, the second and third response was from the server while just the first request got response from WMFS.  This showed, though first request fails, the second and third requests are crucial.

Beyond the corners of the problem

I see, there is no harm in having WMFS in place. Customer who procures the product can have WMFS in place. Product cannot say, not to use WMFS to its users.

The product (client module) should be able to parse the response and have identity mechanism with server token to parse data, which it receives be it from server and other sources.

Learning

For engineering team, it is an alert to handle data in all possibly identified corners. To me, it was the learning which said, not to be negligent and hand over the investigation if you do not request for the update after few minutes.

To business team, the learning is to follow up on investigation which started and see a closure.

Android: Test investigating for RAM consumption - Bit more on Garbage Collection

From this post on knowing about the heap, we moved to this post which gave brief thought on the GC. Continuing the understanding of heap, I will share what I have gathered for garbage collection's algorithms.  It is upto to choice of people to select and implement algorithm(s) and technique(s) in the JVM they build.  To remind, Android OS has Dalvik (till 4.x) and ART (from 5.x) JVMs. For understanding and carrying out the test investigation for RAM consumption, being aware of the GC algorithm and techniques is a plus point.

Roots and Garbage Collection

The GC algorithm will be doing these two things.

1. It must find the dead objects and live objects
2. It must reclaim the heap space used by the dead objects and make it available for application/program

Finding the dead objects is accomplished by identifying set of root and enumerating it to determine the reachability from the roots. An object is reachable if there is a path of references from the roots by which the program/application can access the object. The reachable objects are called 'live' objects and those which are not reachable is called 'dead' object i.e. the garbage.  Any objects referred by a live objects is also reachable and it is 'live' object.

What may be the GC algorithm chosen in the JVM, the JVM will stop the running application when it has to run the GC. When this happens, every thread except for the threads needed for the GC will stop their task for a while. The interrupted threads will resume only after the GC is completed.

So there is a need for fine tuning the algorithm and techniques of GC used in the JVM, such that the interrupted time will be as small as possible. Dalvik and ART are no exception from this.

Source of the "Roots" in JVM

The executing program/application will have access to root, always. The root set in a JVM will be based on the implementation. Few of the source of roots can be as below

• object references in local variables
• operand stack of any stack frame
• object references in any class variables
• strings stored in the heap (from loaded classes) -- for example, class name, method name, field name

Approach to identify live objects among dead objects

Reference counting and Tracing are the two common approach for identifying the live objects (reachable) from the dead objects (not reachable) in the heap.

The reference counting GC differentiates the live objects from the dead objects by keeping a count for each object in the heap.  This count keep the track of numbers of references to that object. The tracing GC will trace out the graph of references starting from the root nodes.  The objects that are encountered during the trace, will be marked and it is known be live object. The one which are not marked in the tracing are the dead objects and needed to removed from the heap.

Further learning and exploring

It will be not in scope to write more about the different algorithms and techniques with this blog post. However, use the below to learn more about the GC algorithm, types and techniques. It will help in knowing how the ART better.

1. Generational Collectors -- Weak generational hypothesis; Young generation; Old generation; Permanent generation; 
2. Serial GC
3. Parallel GC
4. Parallel Compacting GC
5. Concurrent Mark and Sweep GC
6. Garbage First GC
7. Reference counting collectors
8. Tracking collectors
9. Compacting collectors
10. Copying Collectors
11. Adaptive Collectors
12. Remembered sets and popular objects
13. Reference objects -- reference object and its referent
14. Reachability state changes and use of Cache, Canonicalizing mapping, and Pre-Mortem clean up

Android: Test investigating for RAM consumption - Garbage Collection

Getting the brief information on heap this post, I will move on to Garbage Collection in JVM. I have tried to understand Garbage Collection and here is the brief information on the same.

Garbage Collection

When an object is no longer referenced by a program, the heap space it occupies it can be recycled so the space is made available for subsequent new objects.

The JVM has to to keep tracking which objects are being referenced by the program which is executed, and finalize and free unreferenced objects at run time.  This activity will need more CPU time than what it requires when program explicitly free unnecessarily occupied memory.  How to optimize the CPU usage by the program is key in designing of the product so product can still do better when CPU is loaded in a given context.

Heap Fragmentation and Garbage Collection

The heap fragmentation occurs gradually as the program execution happens. The new objects are allocated and the unreferenced objects are freed. The freed portions of heap memory are left in between portions occupied by live objects. In this case, the request to allocate new objects will be filled by extending the size of the heap even though there is sufficient unused total space in the existing heap.  And this is possible if there is sufficient contiguous free heap space available into which the newly created object can fit.

The swapping is required to service the growing heap and this can degrade the performance of the program which is in execution.  Now, I will leave it to you to consider what and how will the impact be from heap fragmentation in desktop and mobile device, running on lower RAM. Have you ever noticed message saying "Out Of Memory" or "running out of memory"?

With the garbage collected dead objects, the problem is, it can impact the performance of the application/program. The JVM has to keep track of the objects being referenced by the executing program/application and finalize the unreferenced objects (dead objects) in run time. All this happens on the fly while one uses the program/application.  This activity can consume the CPU scheduling time. What if the program/application itself freed the memory? Won't it consume the CPU, then? Both consumes but how much is the question. It depends on the code of the application/program and the hardware as well. I'm not sure if one can handle the CPU scheduling in garbage collection process to free the dead objects. If it is possible to handle the CPU scheduling in this context, programmers should have the constraints and limitations; else by this time most of the app would have been better in GC environment if the CPU scheduling was in full control to the written code of program/app.



Graphical Representation of GC and Heap Fragmentation

Figure-1: Garbage collection contextual elements

Figure-2: Heap Fragmentation from garbage collection

Android: Test investigating for RAM consumption - Heap and Heap Dump

The process of allocating new objects and removing the unused objects to make space for the new allocation is termed as Memory Management.  

Object Memory Management - Heap and Stack

Hearing the two terms Stack and Heap, can bring confusion in the memory management. Though I will not get in detail about each here, I will share briefly for what I have understood.  

The 'stack' memory is used for static memory allocation and 'heap' is used for dynamic allocation. For those allocated on the 'stack', they are stored directly to the memory and the allocation happens when the program is compiled. Whereas for those allocated on 'heap' have their memory allocated at the run time and the size of heap is limited by the virtual memory. Both the 'heap' and 'stack' are stored in RAM.

Shallow Heap and Retained Heap

Moving ahead, in 'heap' we will hear the words -- Shallow Heap and Retained Heap.

• Shallow heap -- is the amount memory consumed by one object. That is, the amount of memory allocated to store the object itself by not taking the reference objects into consideration. The shallow heap size of set of objects is the sum of shallow sizes all objects in the set.
• Retained heap -- is the amount memory consumed by all objects that are still alive by objects at the root of the retained set. The retained heap size of an object is, "shallow heap size" added with "shallow heap size of objects that are accessible directly or indirectly, only from this object".
• Retained set -- is the set of objects which will be removed by garbage collection when an object or multiple objects is garbage collected.

In other way, the shallow heap of an object is the size of object in the heap. The retained size of the same object is the amount of heap memory that is freed when this object is garbage collected.

From these two resources (1 and 2), I have understood the details of Shallow Heap and Retained Heap. I feel it is simple and well explained; I have been referring it since years.

Heap dump

It is a snapshot of the memory at a Java process at a given point of time. The dump will consist of Java objects and classes. The reference of memory for the objects and classes will be available in the dump taken. If the Garbage Collection occurs at time of obtaining the dump, then the obtained heap dump will have details and information about the remaining (existing & used, existing & unused) objects and classes.

The heap dump do not tell who created the object and where it was created. Based on type of heap dump, the information available in the dump will be as below.

1. Classes -- Classloader, name, super class, static fields
2. Objects -- Class, primitive value and references, fields
3. Thread Stacks and variables -- The information about call stack of threads at time of obtaining heap dump. The information about local variables and objects.
4. GC Roots -- The mentioning of objects which are reachable by JVM

Benefits of sharing the debug APK with tester

On knowing -- the myths around debug APK; what is debugging? and how tester can debug and be of help to programmer?, I tell about the benefits.  The programmer, product owner and team will benefit by sharing the debug APK to tester for testing. However, make sure you are in synchronous and mutual agreement with the tester if there is a 'concern' in sharing so.

"benefits" to programmer in sharing Debug APK to tester

Test investigating using log

With the debug APK the code written to print system messages, debug messages, and stack trace will print the information into the log.  While testing, tester can learn the logs to monitor -- what's happening within the device; what's happening to the app and what's happening in the environment where the device is. Won't this help to debug and investigate the problem or incident? The additional information shared by the tester apart from the incident won't it help programmer to the trigger or cause of the incident?

Test investigating with variables and breakpoints

The source code of app is available to tester and tester is using the debug app on her/his device or emulator. During this time tester can watch the variables, execution sequence and flows, knowing the stack details around the break points and analyze the code. This will help the tester to be stronger and get skilled each time. As a return, it comes back in testing to your product/app. Getting to frame level debugging and testing, wow! Is not that a value add?  If the tester need assistance to start here, giving few hours initially is a worth investment!

Tracking and testing for consumption and allocation

Yes, testing for allocations and use of HEAP. Further testing for network consumption, battery consumption, CPU consumption and UI performance. Is this not a value add in building high performance mobile app?

"limitation" with benefits in context of tester

As the benefit, there are also limitations in what a tester can do in context.  One such context is tester not having the source code of the app but has the debug APK.

Here, the tester might not be able to look into variables, stack values and test using the breakpoints. But, the tester can decompile the APK and work with compiled code with breakpoints and watch the execution flows. This is time consuming task and the task which can end abruptly if insertion of breakpoint and flow is paused incorrectly. Reporting the problem out of this observation is not suggested unless it is very sure that there is a problem.  But this not stop from testing for consumption and allocation by the app on the device.

As this, there can arise several context which yields the limitation. Figuring out how better the testability can be visualized and making use of it in testing, is the skill of tester.  Everyone in the team will have role to help tester to acquire and hone such skills.

"why" the delay in giving a debug APK to your tester?

If you are a programmer and product owner, then on knowing these benefits, yet if you are delaying to share a debug APK with tester, then there is a concern which needs to be addressed by project management.  This is what I learn.

If you are a tester and have come to know the value you can add from your testing, it will be a unfair to be quite and not asking for the debug APK.

"closing" thought with iOS and WP debug app

As with Android APK, the iOS and WP apps also have debug apps.  The benefit of it is quite similar. As well, there are additional benefits when testing in the respective programming environment with it's devices with the debug app.

Explore the benefits. Encourage the better and useful testing for the context of product.

Debugging and the debug APK

As a textbook or reference book reader, debugging is "the process of identifying and fixing of problem that stopped the product to work as intended."

"debugging" - as how I see

As a practicing tester, I see, "debugging is a empirical activity which involves investigation, reasoning and analysis on the buggy/risky/enhancement incident to learn the details about it, so that a fix solution will be found for it." 

There are different techniques in debugging and the person doing the debugging may carry out multiple activities in parallel.  Discussing the different techniques, approaches and activities in debugging is not with scope of this writing. Hence I'm not sharing much about it, in this post.

I learn, "having the debugging attribute in the product is testability strengthening factor."

"no!" to debug APK while USB Debugging is enabled?

In my testing practice, I test for mobile apps as well. I test Android, iOS and WP apps.  In the context of this blog post writing, I will stick on to Android app and Android APK.

It is very common for a programmer to have "USB Debugging" option enabled on her/his Android device by turning on 'Developer' permission. So the programmer will be able to install the Android app while it is being developed. Further the programmer will use this installed app for testing and debugging, typically.

When tester enable 'Developer' permission and choose "USB Debugging" option to assist in debugging the problems in app, won't that help in developing app? Won't it reduce the time and efforts of the programmer in analyzing the problem when the tester write the detailed Test Investigation report from the debugging? To a tester, who aspire to be skilled mobile app tester, it is very usual to have this setup on her/his test or personal device as the programmer will have.

If so, then have you ever said "no!" to share the debug app to your tester who has same interest in the product as the programmer and product owner?  Why? I will leave it to you for introspection.

"few" myths around sharing the debug APK

There are myths which gives fear in mind when asked for the debug APK. I have tired to demystify few of them here and help to be confident in sharing the debug APK with your tester on mutual understanding and agreement in common interest.

"types" of debugging in developing Android APK

I understand, the IDE used to program the Android APK will use the Java debug by default. That means, the code compiled in debug mode will generate the APK which works with tools based out of Java to see the internals of app.

Along with this, if the IDE supports different configurations of debugging, it can be used. I have not used apart from 'Java' type so far. Hence I cannot share anything further on this.

By the way, when I say "developing", I mean programming along with testing.


"differences" between the debug APK and non-debug APK

When said non-debug APK, the assumption is that it will be a release APK. The debug APK is for the testing purpose. In practice, the release APK will have debug attribute to false. Beside this, the certificates used to sign the APK will be different when it is debug APK and the release APK. These certificates will be bound to date for expiry.

Both debug and non-debug APKs can be ZIP aligned and signed.  Making sure the log writing code is available in the app is essential to print system messages, debug messages and stack trace.

next Post

In the next blog post, I will share the benefits what the programmer and interested stakeholder can get by sharing the debug APK with the tester.

Few myths around sharing the debug APK

The debug APK is a useful build which helps to learn the internal aspects of the app. If security is a concern with the debug APK, then it has to be handled in building the security layers for the app and it's code. Saying the debug APK is the gateway to security problems is not thoughtful, technically and logically.

Here are few myths and fear that is observed when asked for the Android debug APK.

Myth-1: Given the release build/app it cannot be turned to debug build/app

One can use the release build/app and turn it to the debug build though the programmer do not share one. However, there is no assurance that this converted debug build will be functional as the release build/app. If tester converts it to debug build and reports the behavior which are not valid then it is additional effort to programmer and project to triage and then say it is not valid. This happens!

Myth-2: One will have access to the code of debug app

One will have access to code when one installs the app on her/his device. The question is in what format and how the access will be available. Apps having the obfuscation with help of ProGaurd and DexGaurd will obfuscate the compiled code of app. To deobfuscate it, one will require the mapping file key. Further one can go and ahead and work to deobfuscate with different approaches and techniques with no mapping file available as well.  It is upto what level the security is strengthened for app.  Anyways the code is available (in a format) when one installs the release APK/app. 

Myth-3: One can tamper the debug app and use it illegally

From above said myth's opinion, I will continue for this topic.  While building the release APK, the programmer will sign the APK with certificate. The release build certificate and debug build certificate are not same. The person who wants to use tamper the app, still she/he can decompile, edit the app, deobfuscate, recompile, sign the app and build it. When signing on recompile, if the person do not have valid certificate still she/he can go ahead with self created test certificate.  But, the question is will this certificate work well for the app which has multiple legacy SDKs included and expecting a certificate and key. In my test practice, I have seen, the app failing to function for its purpose though it launches in such context.

Myth-4: One can misuse and learn the sensitive area and information from debug app

With out the actual source code, one can reach to stage where she/he can see the register and stack values by using just the debug app's compiled code. For getting into this level, one has to successfully decompile and build it back to SMALI data structure and then start the debugging to look into the stack registers and flow sequence. If the obfuscation is applied at this level, then it a round more of strengthening. Yet it does not mean one cannot cross it but the time needed is unknown.  This is possible with non-debug app i.e. release APK as well.

Myth-5: One can know what happens in the debug app while using it

The very purpose of the debug app is this i.e. to know whats happening when using the app. The question is -- who is using the debug app? Without the actual source code of the app, the information learning will become narrow as the testing progresses with debug app.  

Myth-6: Giving the debug is like giving the permission to code of app

Debug APK means not permission to code of app for editing and building.  If there are other checks on the decompile, recompile and signing to stop app from functioning after this, the APK will not be usable as the release APK.

Myth-7: One can gain the control over the app with debug app

Unless the decompiled app is edited in code and built again to do illegal and incorrect activities, it is unfair to think as this. If this is the possibility with the Android app which is being built, then it has to be handled in how the structuring happens for security layers of the APK. This is nothing to do with debug APK.

If the debug app is pointing to the staging or test environment and access is restricted with IP white listing and similar other approaches, it will restrict the access from public use. The other way is to reduce the validity time for debug signing certificate. Keeping this validity that is reasonable for the test cycle is another approach. After this validity the debug APK will not get installed on devices unless it is decompiled and resigned again. 

API Testing -- HTTP and REST, the difference exist

From the last post in this series, I'm continuing to share my learning and till day practice in testing the APIs. In this context, this post will have information on below mentioned.

• HTTP and REST, the difference exist

I wish to keep the below topics in the next post so that it helps in gaining the clarity of RESTful APIs with different protocols.

• The usual in the WEB API testing
• The "not thought about" in testing APIs -- Web and RESTful

HTTP and REST, the difference exist

HTTP is a protocol and one of the way to transfer the files. This will connect the pages in the World Wide Web. While the REST is set of constraints documented for the architectural style for web. This constraints are to ensure the scalability and ease use of the web systems.

The common understanding for an engineer is the web runs on HTTP and so the RESTful API.Yes, the REST API will use and can use HTTP as communicating and transfer protocol. This understanding is common because one might see HTTP being used often and most times with RESTful APIs. But it is not mandate that RESTful API should always use HTTP only.  The RESTful API can also be for FTP, SMTP and other protocols to transfer and communicate.

Now let me reiterate this line again from above paragraph,

"While the REST is set of constraints documented for the architectural style for web"

The web has many protocols to communicate and transfer; one among them is the HTTP. So now we know the idea that RESTful API can be designed and used for any communicating and transfer protocols other than HTTP.

Then the difference between REST and HTTP to mention here in short is as below:

1. HTTP API is the any API which makes use of HTTP as its protocol.
2. RESTful API sticks on to all REST constraints which are described in the dissertation of Roy Fielding.

Usually the 'Web API' is referred to HTTP API. However it is good to seek clarity when one hears "it is the Web API".  That means, SOAP can also be HTTP API if it uses HTTP. If the SOAP can use HTTP for transfer and communication, then RESTful API are possible for SOAP as well over HTTP, isn't it?  This looks to be tricky in understanding but it is not. Spending sometime to ponder and explore on this area, will give the clarity in learning this.

IME Test Model: Briefing the "RICH VIP MUST PLUG AND HE PUTS LOCK"

On posting this, I had not covered what this mnemonic means.  In this post, I will cover briefly about each factors that comprises this mnemonic. An IME app do not necessarily have to be an app always which will be installed on the mobile phone by the user. It can be a SDK which will be bundled in to the OEM as well.  Testing an IME app and testing the IME's SDK is similar? If wished to share your thoughts and ideas to program and test the same, please do share.

What is this mnemonic ?

RICH VIP MUST PLUG AND HE PUTS LOCK

Before briefing it out, I want to tell -- this will help a programmer, tester, product owner or anyone who is interested in building one such mobile application. It helps by being a pointer to provide an idea to categorize the development (programming + testing + segmenting the app to market & users) of IME. Thereby, it becomes easier to assess them technically as well for the context. I can derive the ideas to test the IME and IME's SDK technically from this test model and assist in the better development of it.

1. Response 
2. Interruption
3. Content & Key Layouts
4. HIG
5. View
6. Intent Search & Discover
7. Permission
8. Messages
9. UI
10. Settings
11. Type of IME
12. Platform & Specific Device
13. Languages
14. Users
15. Gestures
16. Action
17. Notification
18. Dictionary
19. Help
20. Error Handling
21. Prediction
22. Updates
23. Trails
24. Sound
25. Locale
26. Orientation
27. Cursor
28. Keys

Briefing the 28's of IME Test Model

In this section, I will convey few lines among what I have thought about each factors while I was identifying and picked it.  To make it clear, this cannot be treated solely as the test ideas and design ideas for programming; rather, it will be a pointer to start and influence in building the ideas to test and program.

• Response -- Anywhere in the IME app and where it is used in conjunction with other apps, you have a thought how 'fast' or 'slow' is the response, then find the tests and know how to do it. Anyways, what is that I'm going to get by learning this? Know it!

• Interruption -- Not the small one for sure. What is the interruption to you and what can be part of interruption in mobile technology? It is beyond the calls, messages and the other common words you and I have heard. How good and bad the IME app can be on the device if this is handled well? This can be a good start after knowing what is an interruption.

• Content & Key Layouts -- It looks simple yet it is not the simplest by the way. If the IME is built for global users or specific to region, what is that I need to have in the content and key layouts?  Right from visual design to interaction design to data architecture to mapping it to the mobile hardware with locale selected, is for sure a interesting programming and testing. Think what's in your ideas to test and automate it.

• HIG -- What do the platform's HIG guideline tell about design pattern, IME design and what you unlearn and learn about the design you were visualizing, seeing and wanting? Having the reference of this definitely helps when you want users to use the IME you are thinking, programming and testing.  Do you know if your platform allows to build one IME app? Is that right !?

• View -- It is beyond the UI, if you thought it is the UI.  For an app programmer, it can hold her or his breath if the View is not handled. Know the different kinds of Views available in the IME and how to program for it to hold relevant data (re-read Content & Key Layouts) and process it by using it as input and output.

• Intent Search & Discover -- The mobility is evolved and so the Artificial Intelligence with the chat bots and service API integration. Your IME has got this efficiency and feature or wanting to have it? Then know how to consider programming constraint and testing the same. It is no more just the app; it is with-in and from-outside-inside of the app together with AI. Discover your tests to test it.  Hey what's the version of mobile OS version you are targeting to support and build the IME ?

• Permission -- What is the permission you are thinking about to grant as a user for the keypad on your mobile app? Do the permissions you want to grant and that you do not want to grant for IME app - consume your device's resources (why?), slows your interaction with the task you do in the device (why?), and puts you in state of embarrassment and annoying (why?) Then think about the permission for your IME you are building and testing. Wait, are you happy with the permission you ask from the user for using this IME morally, technically and ethically?

• Messages -- What you get  in your thoughts when said the 'message' in an software mobile app? Ponder over and touch Ben Simo's FAILURE heuristic when required consistently. The message looks to be as UI part but it is more from the behind the UI to the UI.  What messages are programmed in the IME app and localized to suite the context of app usage with locale of device? Besides the known error, warning, and info, what is the kind of other message I do not get known often? Is this taken care in the IME app?

• UI -- Not the mastering one for sure when it comes to app that supports different locale and language and several hardware configuration devices. Pattern is the key in keypad. The UI anywhere is not all about the visible UI aesthetics but it is more about the performance and fastness. How is the UI of your app is light and yet straight in delivering optimized expectation? What should be part of programming such UI? What are the tests I should be figuring out and sample to know the UI of the IME app?

• Settings -- How the setting changes of IME add to the factors that handles the device's service and intent? Besides the Settings functionality of the IME app or IME SDK, what I have to consider in programming and testing of it?

• Type of IME -- What's your IME type and how do you input the data? What are the different option it supports to provide the input to mobile device and apps?  How about an SDK here, forget about the IME mobile app. Now list out that should be handled in programming it and how to test the same. Is there a pattern in how the input is provided via IME?

• Platform & Specific Device -- How's the IME built specific to platform? How different can be the behavior of IME when there is a behavior specific to device and its software? How to program for such cases in handling behavior of IME? How to isolate, debug and test such behavior?

• Languages -- Language which IME claim to support, do device support that language? How the language's alphabet characters are rendered and shown? Does the construction of word in language looks similar to as and when human does it? How the mapping of keypad keys happen to language's alphabet character, symbol and other characters? UTF, the common word; but is that has to be considered when programming and testing here?

• Users -- I want to go beyond these - User Personas, User's Device and Accessibility. Not that I want to ignore them; I see there is much more than the above said. How can I identify and capture them in my test?  What is that specific support a user will get from the platform when providing the input, especially via IME? Okay, let me bring this thought here -- most say, they scaled and built the technology and app/product for customer engagement, do this app have that technology segment? This is just one thought. As this, I have much more where I can identify, derive my tests and see which are suitable to automate among them. Just to trigger a chained thought from the previous one, how about A/B strategy to read the user's priorities and mindset while using the IME? Then connecting it to the User Customized IME. How to test it and to have it in a app?

• Gestures -- Do you have list of gestures available and supported on your device and OS? As usual, the Gesture will be used by IME users. But what is that comes with gesture as a cost? How does that impact in device fragmentation space? How the gesture which you have set works in case bilingual and native keypad? Customized gestures do exist and how will I intimated when there is a confusion in gesture recognition by the app and device? To spice up a thought here, just think -- how the gestures make differences when the IME is used with a mobile app to input and when it is used to input on a mobile web as well. Any new gestures available via this IME which is unavailable till date? How to approach testing of gesture on different types of touch and display framework of platform if I'm bundling the IME SDK with OEM?

• Action -- The action keys available in the keypad, how do they get localized as and when the language is chosen? Also how it reflects when the locale of the device is changed.  Besides the functionality of such action keys, what can I consider in programming and testing of such keys? One example at UI level is, what should be the direction of arrows in action keys when it is being used with right to left language?  That was at UI level; if I had to figure out how the performance of the IME will be influenced by such keys, then what should I learn now?

• Notification -- On knowing the types of notification available in each platforms and how it appears, how do an IME app handle and show the notifications. Do this notification need a trigger from the app internals to appear by user action? What is the contribution of it in battery drain if the notification uses the wakelocks?

• Dictionary -- It comes with the size either the user start adding the words to it or the products comes with limited words. Size matters on the mobile phone in space utilization aspect? If user can add the words to the dictionary, then the space utilization grows. What if the IME app supports for bilingual and multilingual? Then space occupied in the dictionary file for different language's word differs? As well the dictionary can be downloaded and gets updated regularly. Do this have any effect on low end devices? What other tests and factors I have to consider when testing for the Dictionary apart for the functionality? Did you forget the the file type and format?

• Help -- In my opinion, the help section of IME is yet to become helpful especially when it comes to non-English language. For example, if the IME supports Indian language, how to use the keypad to type the words? Likewise, if it has to support different languages, then the pattern of using the keys in keypad to type the words in respective language, should be in the help section? This is open to discussion considering the context and who will the targeted audience of the IME app.

• Error Handling -- I will use Ben Simo's FAILURE mnemonic here. I'm learning to use in UI and non-UI mode. Coming to the error handling, I understand 'error' is when a user perform actions which is not valid as per the product and the product has to inform the user without causing any lose to customer in product's context. Few devices may not support languages which is supported by the IME. While such language is used by the user, how will this be handled by the IME app?  This is one example. As this, figuring out the error handling in each factors mentioned here, helps.

• Prediction -- Before I talk about how to test the prediction mechanism, how to bring in assistance of accessibility to the prediction feature in the IME app? Let us think on this. Now, the prediction in-turn builds the size of the dictionary and the predicted candidate words come from the dictionary which I have and building in the device. That said, dictionary and the content in it for the language chosen plays a role! IME app has to learn the user's typing and words usage style in the chosen language. That is not simple but doable and testable. The prediction comes from the user's input to the dictionary; if she or he does it incorrectly then the prediction that appears later will also be incorrect. Understanding the key:word and value:frequency in the algorithm used, helps.  Insertion, deletion, transposition, alternations, and duplicates all will come into picture as user types and sees the prediction in candidate words/list.  How do I program this? It has to be quick in calculation so I see the prediction appearing and changing as I type. How do I test this? What I have look for performance factor when the dictionary size is growing with words for a language which will contribute to the prediction?

• Updates -- When updating the app, do the content of app which is available now will be retained or get cleared? The content of the IME app such as dictionary and languages should be untouched unless the app wants to sanitize and clean it as well. Post update how these contents will be used by the app?  This is one instance of thought among the several what we think while building and testing the IME app.

• Trails -- The UI mark trails left after using keys and swyping over keys, do introduce any problem on the device?  If it is low end device, then such trails of gestures and keypad usage cause CPU and GPU overload?  This is one such thought to consider in programming and testing the IME with trails sign feature.  As this collect and build the list to consider for programming and testing this factor. What about the trails when using the Indian languages and other similar languages?

• Sound -- Build the dictionary file size and test for keypad sound and other that you hear when using the IME app. Do you see any difference in speed and how the sound is heard to you? As this figure out what needs to be considered when programming and testing for the Sound factor with the IME app.

• Locale -- On changing the locale on the device how the IME app changes accordingly? What has to be considered for programming for an IME app with locale? How to test it? For example, I have changed locale to reflect right to left language in the device and toggling between the bilingual in the IME app. How will the IME app have to behave with locale's context and selected mode context in app? As this, build the list to test for locale factor. Interestingly this clubs with most factors here very closely and will have dependency.

• Orientation -- Orientation do have contribution in screen overdrawing, GPU rendering and memory consumed. Apart from this, how do the keypad look and evolve for different languages? How the keys and different UI areas look in the keypad with different language in different orientation of different screen size? This is a start area to explore.  Any key get missed in portrait mode but not in the landscape mode?

• Cursor -- Do this remain consistent with device standard or will it have standard of the IME app? How the cursor assist here in accessibility? For example, the cursor blinks. Is there a case, where the cursor stands still without blinking? How do the IME behaves in that case? Moving of the cursor in the word you are typing or typed, changes the prediction shown? These are few examples to consider in programming and testing of cursor in the IME app.

• Keys -- The visible part of the IME app to a user. Contextualizing the keys is one feature with respect to locale, language chosen and input being entered. How the key has to manage its space within it to show different characters when primary and secondary are enabled? Colors, themes, and Settings leave any influence on the keys? For example, the touch area sensitivity on keys.  As these what are the factors do I need to consider in programming and the testing the keys of IME in different screen size and hardware devices?

Configuring for Go Programming in Windows OS

In this post, I share how to configure for the 'Go' so I can use the environment for helping the testing of me.  The way tester did her/his testing a decade back is still applicable in a way, say, but technology has evolved and changing every day.  Moving along with the technology and being smart and quick in testing is expected today.

In one such regard, configuring the 'Go' for setup can be confusing to a tester most times.  I share how to configure the Go setup. So a tester will be ready to make use of it to compile and run the libraries to boost the testing based on where the context needs it. This post illustrate how to configure the Go on Windows OS.

While I'm writing this post, the link is valid and available. I can download the Go's MSI file from this page. Following the installation instructions in installer and on successful installation, I need to configure and save the environment variables in Windows OS for 

• GOROOT -- pointing the path where the Go is installed on the Windows OS
• GOPATH -- pointing the path where workspace is setup for Go

For the workspace of Go, I need to make sure it has got three directories -- bin, src, and pkg under the root directory. Example, if the path is d:\go_workspace, then under the directory go_workspace three directories will be available i.e. bin, src and pkg.

I assume that Go is installed and configured as said till here. Now, open the command prompt and provide this command.

go version

Doing this, should be able to see the version of installed Go.  At this time of writing, I see as below.

C:\Users\Pc-Ravisuriya>go version
go version go1.7.1 windows/amd64

Get the sample program from the Git of Go by providing below command in the command prompt.

go get github.com/golang/example/hello

 Now I can execute the hello program from where I'm (but I have to provide the path of program file for running it) or I have to go for the folder where the hello.go program is available. In the above said workspace path example, it will be under, d:\go_workspace\src\github.com\golang\example\hello.

Going to this directory in command prompt and running below command, I will see,

d:\go_workspace\src\github.com\golang\example\hello>go run hello.go
Hello, Go examples!

API Testing -- A Web API transforming to be the RESTful API and Web Services

After knowing lines behind in birth of the REST, I will move on to know more about the Web API and what does the REST architectural style have to contribute for it.

What is a Web API ?

I keep it simple as this -- The application programming interface available to facilitate the service to the needed client (also server) over the Web. It provides the interface to data and service methods for who are in need of it.

Note: Refer this comment to have short briefing about what is an API.

Here is what MDN (Mozilla Developer Network) says and Micrsoft Developer Network say about it.

What is a Web Service ?

Web services are web servers built on specific purpose which supports the needs of a site or other applications.  The client program will use the APIs to communicate with the web services.  The API will expose data and functions to facilitate communication between the applications by allowing them to exchange the information. For example, the payment gateway page opening to Net Banking channel and taking to the login page of the chosen bank, is via API offered by that banking web service. So the credit card payment and debit card payment too.

Here is how, the client, API and web services placement look in general.

{ Client } < - - -  request and response - - - > { ( Web API ) <---->( Web Services ) }

REST API - Is this right way of calling it ?

The design of Web APIs adhering to the REST architectural style for having the access to the web services and it's technologies, is called to be REST API.  But the question is, should I call the REST as an API ?  

No, the REST is the architectural style for the Web and a concept; it is not an API.  An API having these essence of REST is called by name REST API.  

Personally, I feel, the right way of calling it being a Software Testing practitioner is,

• REST conforming APIs
• or, APIs conforming to REST architectural style

What is RESTful API ?

An API is beneficial and usable, when there is a web service in back which is reliable and accurate to it's purpose.  If there is no web service as this, then API is still a gateway but not serving the actual purpose for which the API is used.

The web service becomes more powerful with the APIs conforming to the REST architectural style. When this happens, it will be RESTful Web Services.  In other view, the APIs having this efficiency is also RESTful by conforming to it's standards and practice.

Next in this blog series

1. Usually in the Web API testing !?
2. The "not thought about" in RESTful API testing

API Testing - From knowing what's an API to the time that saw birth of REST

It seems to be uncommon if said I do not know what is a function, library, class, protocol and binaries in the Computer Science & Engineering.  Extending to this, it is more likely and usual to hear the word, the API - Application Programming Interface.

Have you read this which is so common if you are building and testing the APIs?

What is an API ?

Before walking into this discussion, let me try to understand what is an interface! Yes, an interface. Let me split this word and see what I see.

I see, "inter" and "face".  That is, there is a face between the inter -- which can be one or more.  A face as a property and attributes, isn't it?  So when I have an option to inter look in the face and take what's available there, I say, -- "I need it or I need some of these which can help me in ....."

From an API, I need it or I need some of these which can help me in doing what I want to do in the software system environment.  Further learning what is -- Application Programming, it is the programming of an application what is put in the environment to do the expected on purpose.

Now this Application Programming having an Interface so there can be a communication with methods or using the methods (functions) and it's associative elements. Upon communication, transfer of the messages i.e. data happens between within the system and also between the systems.

An API is (set of) programming instructions that is used to build the software application.  The context defines, what is the API there and how it will be used.  This further classifies the APIs now into types and it is briefed below.

Types of API

An API can be everywhere and to be so, it can be in the below said contextual forms.

1. Software Library API
2. Hardware System API
3. Web based API
4. Operating System API

Each of above said types further classified much more and in detail. Overall, in general these types covers most cases as I understand at this point of time in my practice. If you happen to see and using it beyond this, kindly let me know; it will help me to learn.

The API which aids to talk between two and more technologies or within the technologies, helps to pick what is needed alone and work on it. The encapsulation and interface idea of Object Oriented Programming can be seen in concept of the APIs and in the programming of it.

REST Architectural Design for the Web

All of sudden it appears like the word 'REST' is so new in the technology world, when one hears it for the first time and subsequently.  Well, the word REST in the Web exist since year 2000 and around somewhere.

Going bit more back in years, in the December of 1990, Tim Berners-Lee started a non-profit software project called World Wide Web to facilitate the sharing of knowledge. Working for a year, he had invented and implemented the below.

1. URI - Uniform Resource Identifier
2. HTTP - HyperText Transfer Protocol
3. HTML - HyperText Mark-up Langauge
4. The first web server - it is still up and running at http://info.cern.ch
5. The first web browser which Tim Berners-Lee named as "World Wide Web" and he renamed it as 'Nexus' later to avoid the confusion with the web.
6. The first WYSIWYG HTML editor - it was built right into the browser.  WYSIWYG is an acronym for What You See Is What You Get.

In 1991, Tim Berners-Lee wrote on the Web's first page as below

The WorldWideWeb (W3) is a wide-area hypermedia information retrieval initiative aiming to give universal access to a large universe of documents.

Since then, the Web took it's growth rapidly. In 1998, as I have witnessed, the results of exam were published on the web by the education board in state Karnataka. Then the chat room asking 'ALS please' and as this.  Today while I write this, there is a Big Billion Day and Grand Annual Sale happening by the retail competitors of India where the customer purchases over the web via website and mobile app, thereby making the crores of rupees transactions in a day for business.

Getting back to 1990's, Roy Fielding, co-founder of the Apache HTTP Server Project was concerned for the Web's scalability problem in 1993. He and his fellow practitioners, observed Web's scalability constraint and came forward to improve it.

Roy Fielding grouped the constraints into six categories and he referred them as the Web's architectural style and it is as below:

1. Client Server
2. Uniform Interface
3. Layered system
4. Cache
5. Code-on-demand
6. Stateless

Along with Tim Berners-Lee and others, Roy Fielding worked to increase the Web's scalability and wrote the specification for the new version of HTTP 1.1.  These standards were adopted across the Web and contributed to the growth of World Wide Web.

Hey, you know, if HTTP is used in the software application you are building, the skilled practitioner says to use HTTP 1.1, so the caching techniques can be derived on using it upon mentioning it. The network attribute optimization, you see!

On avoiding the Web's scalability problem, Roy Fielding named and described the Web's architectural style in his PhD dissertation, in year 2000. He gave the name Representational State Transfer (REST) to his description of the Web's architectural style which is composed of above said constraints.

REpresentational State Transfer (REST) -- is the name of the description or derivation of the Web's architectural style.

What is in the next post? In the next post of this series, I will walk through the thought of

1. Web API
2. Restful API
3. REST API -- Is this a right way of saying it ?

API Testing -- Did you come across these questions?

Do you test API? What tools you use to test API? Hey, you also test the REST APIs? Automate and run the suite that is sufficient to APIs, isn't it? Have you asked any of these questions? Other person have asked you any of these questions? Or, you heard such conversations?

I will share my learning what I'm making by testing the APIs. Use the label "APIs"in this blog for finding related posts to it.

When I hear above said question, the first question that I ask is, "What do you want to accomplish and know by testing the APIs which you are talking about?".  I see no response coming back most times. Yes, we all use APIs! And, we will continue to use! What is that I want to know from testing the APIs?  How do I test? Why at all I have to test for APIs?

I got notification on my mobile phone from the app I have installed?  The APIs helped to receive that notification.  Did I see the stock market's number updating on my desktop or mobile app? That was with the help of APIs. Do I see anyone monitoring the health and fitness with the wearable device and then transferring that data to mobile or desktop? That happened with help of the API.

In the next post, I will be writing the below mentioned.

• What is an API ?
• The types of API
• REST architectural design for the Web

IME Test Model - RICH VIP MUST PLUG AND HE PUTS LOCK

I was approached by my fellow Software Testing practitioners - Shristy and Suchismita for knowing and to have better structure for testing the IME - Input Method Editor.  On listening to their context of current practice and what they wanted to know by testing, I learned for first they need the essential design components of today's IME.

I had to make sure that, this learning is fair enough to start and from here they can assist themselves. On brainstorming together for few minutes, we learned, it is good for a start to have the key integral design components of IME. Have this, the testing can be channeled well in those areas as and how the context demands on priority.

Now we had listed down the IME design components fairly sufficing to start. What are the tests to be done on the app under the IME design components? It depends on the context of testing.  The testers here were able to identify the tests based on the context needs. But the challenge for them was -- knowing how to approach and categorize the IME app.  Now it is addressed with this IME Test Model, tester can help quickly to visualize and pick up the tests under the hood of respective design component of IME.

Credits are for Shristy and Suchismita for pairing up with me and in framing the mnemonic of this Test Model and categorization of it.  This can be referred and used for Android IME and iOS IME apps testing.

A Test Design being guided by the Test Strategy

In this post, this image looks to have too much structures around it.  But, it is not. It is very simple one as I see it.  It is simple when I happen to understand the Test Strategy and what is the help out of it.  Having said this, why is that simple, because, it is from the fundamentals and nothing else.

If I understand, Test Strategy is set of ideas which directs my test ideas and test execution, then the test strategy has influence on Test Design.  Where the test design consists of these elements necessarily within it and it is as below.

1. Test Model
2. Knowing the Oracle
3. Knowing what is my Test Coverage
4. How do I operate my tests and execution of it.

The below picture says the same in the equating form.

The same applies when I have to figure out how to approach the automation for assisting my testing. I use this and when it requires a change in the context, I will do it accordingly and try to get most out of it.

Update (on 23rd Aug 2016):  I unlearn and learn the equation represented as this is more meaningful:

Test Strategy = { Ideas <--> Test Design + Test Execution}

As the test design and test execution in turn helps the ideas to evolve better as the testing progresses.

Inside Out of Test Design: Test Technique Identification with help of Test Model and Strategy -- Part 2

From the overlapping, I move ahead to learn how the test, technique and design is part of each other, I see, the three more now by adding to conversation.  It is -- Coverage, Action, Information (Potential Problem), and Assessment. This looks to be me as in the below image. 

Figure: Test Design and Identification of Test Technique

I understand this for Test Model -- a representation by which I'm learning the product either associatively or relatively or by both means. Further, I can still refine the model to specific learning i.e. to assist in obtaining the specific type of information from my testing.

Moving ahead, I will learn what information is expected from my testing. Knowing this, I will have to build a Test Model or use a existing one, to understand the product and my tests which I will be executing. The Test Model will help me to learn critical factors associated to product, environment, project and related components.

Now, I will have to choose the test techniques or build one. This has to assist me in building and strengthening tests by involving the heuristic within it also by identifying the heuristic from Test Model and tests.  This heuristic will help me to understand the outcome of test.

When I have figured out (or as I figuring out) the Test Model, Test Technique, and Heuristic for the test, I will have know what is the Test Coverage to be accomplished in context of testing.  This gives the dimensions and influencing description to say why is this my Test Strategy, and why I have made use of this model, technique, and heuristic to arrive at this coverage. As I execute my test and the way in which I record it, this overall constitutes the part of my Test Design.

Looking at the techniques, I have known there classification from the Software Testing books as Black Box, White Box and Grey Box. I'm learning there is much more within each of these types than the books say.

To summarize my so far learning, I see, the Test Techniques is one of the outcome of Test Strategy. The test strategy directs me to pick or build a Test Model. With the help of Test Model(s), I will execute the test by learning how I have to execute it and to what extent using the technique(s) (which is also a heuristic).

Inside Out of Test Design: When each disguises each other - Test, Design, and Techniques -- Part 1

I have been and continuing to be mesmerized about the Design Patterns in programming space. While I'm continuing to look at the Design Pattern, I ask myself is there anything similar in the 'software testing' space? If software programming and software testing are subset of a super set, the engineering, then design pattern and test design are to each other. That is, the both exist to support each other well enough in the context. Isn't it?  This is the question among lot other which runs in me when I think about Test Design.

Further, I go into my thoughts (of today) for letting know myself what I understand for 'test', 'design' and 'techniques', I hear below from me.

• test -- An experiment which is an open ended and whose outcome is unknown.
• design --  The set of ideas which says how to (I) use it or how to go about; and how I see it further to use it by learning. How simple it is to use for me, those are the attributes of design with the learnability in context being prominent.  In other ways, it comprises of pattern as soon as I say, 'design'.  Each design has a pattern isn't it?
• technique --  applying; i.e. a skill being applied to study, observe and evaluate a subject of interest further.

While I'm learning this, do I see a familiarity or the familiarities in these three above said?  May be this familiarity or these familiarities, it disguises each other while they are distinct and co-exist mutually in practicality of Software Testing practice.

Why one cannot see the differences or draw the differences between these other is, a simple fact. Yes, the fact, which is not studied quite often; hence it is a fact.  The unquestioned fact is, all three of these are heuristics - which helps in discovering, learning and solving, while it can fail as well.

A test does not help one to learn, discover and solve? But how a test can fail? Test never fails, it provides the information and I tag the label of true positive or false positive to the outcome of it. Likewise, is the design and technique, it helps in discovering, learning and solving; and as well it can fail too. But how do I see the 'fail' in case of design and technique is -- it did not meet the purpose or serve the purpose. Because out of the design and technique, I expect the discrete value which is of usabale, reusable and valuable. Whereas for a test, I see the information is an outcome from the execution of it (or out of not execution as well; not testing is as well a useful test in a context).  This is where the disguising appears by each others to each other.