The Automation Strategy Problem; Not a Appium Challenge

 

In The Test Tribe's forum, I read the post which described the problem as in the below paragraphs and picture.  On looking into it, I learned this can be made as a blog post that tells a strategy for automation.  

Maybe, 10 years back I would have asked the same question.  That's a learning curve.  Today as well, I end up in thinking for a while asking self -- how to test it and how to automate it.  

I want to share how this problem can be looked at from the perspective of testing and automation, and then approach it to automate.

 

Folks. I've two issues on Appium automation which needs your help. 

1. I'm working on a ecommerce website where a payment method is integrated (lets take the example as PhonePe). When i try to place the order in the mobile website with payment method as PhonePe, the payment method app will be opened and I've to complete the payment using it and I'm navigated back to the browser. Issue is - How can i switch context between the mobile browser and the app? I tried using driver.startActivity() but on performing any other actions, it errors out. 

2. Since i need to use the browser to place order and the payment using the payment app, I tried to set up the driver instance with browserName and app as the desired capabilities together. But on running the test, it errors out - browserName and app can't be used together. How can i approach this problem? Anyone who has automated such flows?

Apologies, i'm pretty new to Appium and so, please excuse my ignorance.

Picture: Problem Statement - Description of Scenario & Challenge

Understanding the Scenario and Functional Flow

I observe the below in the said scenario:

1. It is a website; it also has a mobile website
2. It has got a payment option integrated
3. The Appium's Desired Capabilities defined has browserName and the app
• borwserName -- name of the mobile web browser used in automation; it is an empty string if automating an app
• app -- the path of an app to be automated
4. When using a mobile website on a mobile device -- assuming it a mobile web app
• On selecting a payment app -- assuming it a native app
• The context changes to payment app UI
• On completing the payment, the context changes to the mobile website

Challenges Described in the Funtional Flow

I see these as challenges:

• How to handle this said scenario in automation using Appium?
• How to switch context between mobile browser and the mobile app?
• Using driver.startActivity(), it yields an error on performing any other actions
• On making any actions on UI after using the above said method, the error is observed
• Reading the description, it is said that the error is thrown when running the automation
• And, when changing the context back to mobile website from payment app

The driver.startActivity(), takes two arguments -- app's package name and activity to be started.  What's passed for the package name and activity name is not clear from the problem description.  

If the mobile browser is used to launch the mobile website and mimic the action, what is passed as app's package name and activity in driver.startActivity() ? This is not mentioned and unclear to me.

Also what is mentioned for the browserName and app in desired capabilities is not clear.

A Common Use Case

In recent years this is a common use case in a mobile native app having a web view and the websites that have payment transactions.  For example, in the native app when making payment, the web view of payment gateway that shows list of payment choices.  On successful payment, the view switches to native view from web view.

Questions on Reading the Problem Statement:

I have the below questions on reading the problem description:

1. Why did it throw the errors on any actions post calling the driver.startActivity()
• driver.startActivity() will start an Android activity using package name and the activity name
2. The context picked on switching from web view to native and then back to native, is not well picked?
• But it is a mobile website which means it is opened on a mobile browser, right?
• No where it is mentioned as a Hybird app i.e. the mobile website installed as an app
• Does this mobile website maintains its context when switching to a native app (payment app), and then changing the context to (web view) mobile browser?

This takes me to seek clarity for:

1. Is mobile website a installed Hybrid app? Or, is it a regular website which also has a mobile website and accessed on a mobile browser?
2. Is it possible to switch the context of web page from mobile web browser to native app, and vice versa?
• I need to explore it; I'm unsure of it
• When read the desired capabilities, it looks like this can be done
• That is context switching of mobile web browser to native app, and back to mobile browser from native app is possible
• I need to explore on the same to be very sure of it

Code Snippets for Context Switching

Refer to this page for details on using the Web view with Appium.  The below code snippets tell how to find the context of web and native views, and switching to it.

Snippet illustrating the change of context to Web view

Snippet illustrating the change of context to Native view

But, What's Actually the Problem?

If automated as described in problem statement, do we end up in a problem?  I see, yes we will end up in a problem:

1. Need to maintain our automation to make sure it executes the payment app UI anytime
• If the UI of the payment app changes, we need maintain the code
2. Do we have stage environment payment app in this case?
• If we test the mobile website in the stage and make transaction in production payment app,
• Can we continue as this in each test iterations?
• If yes, how long can we continue to use production payment app and pay?
• Will there by any transaction fee charged each time from payment app?
• Can this become a financial cost to the business and client or to stakeholders?
• What other cost should I bear for using this approach?

I need to know:

1. What is that I want to learn from the use case or scenario on automating it?
2. What would be the impact if the test did not help me to learn what I want to learn from automating this use case and scenario?
3. Should I be testing the payment app along with my app? 
• As I write UI automation to handle the web view of payment gateway and then the native payment app, it becomes part of this test.  Should I do that?
4. What information, risk and problem discovery I miss, if I do not automate the payment app flows?
• Is it okay for the business and product, if I miss any information here or if I do not test the flow in payment app?
• How to arrive at this decision?

The decision here need to be rational.  But, being rational alone may not help always.  Can I be reasonable here when I'm deciding or influencing stakeholders when deciding?

This is a Automation Strategy Problem!

If seen, for first this is not a Appium problem.  It is a problem with -- what to automate, how to automate, when to automate, how much to automate, and why automate.   That is, it is a problem with automation strategy on how to approach and execute it.

To me it is a problem to solve with approaching and execution of automation for payment transaction, and not a automation library usage and implementation problem.  

How can I Approach the Automation Here?

I will learn, should this payment scenario be automated on the UI layer for first?  If yes, why?  And, then I will have the below questions

1. Can I use the developer APIs of payment service to test and complete the transaction?
• If yes, then
• Can I use the stage APIs of payment to simulate the transaction flow and its completion?
• If I just use APIs, I will not know what's the functional experience of transactions in native payment app.  Is this okay?
2. I and the product I test, do not have control over the payment system and its apps
• When I have no control over it at any point in time, should I test it as part of my system?  If I did so, should my product as well include the probabilities and complexities of payment system?
• Having this information is good!
• But what can I do with that information?
• Do I have an authority to change or fix payment system with that information?
  
• If yes, good; if no, then the time and resource spent on this s a value return to my stakeholders and their business?
• It is wise to mention that I'm not including and testing the payment system and its transactions as a part of my system
• Because my system does not have a control over payment system in any means
3. If the API that is used for initiating transaction is functional and usable, then I do not have to worry technically from functional perspective of transactions
• We will have to work on -- if the payment initiating web view is functional on my native app and in my website or a mobile website
• From here the control of payment and any transaction problem that arises are in the realm of the payment system
4. In the test report
• I will include the stage payment API request and its response with data
• Talking to payment app organization, we may get the developer API access on stage to test our system on their stage
• Talk to payment app organization!
• Also we can mock the payment API to an extent and in the test report say this is a mock result
• If relied on mock, then we can miss the change in payment system
• I will have the mocking as last approach just to complete a business flow and it will not be my pick unless someone wants to see a business flow completion in a test
• Have a test that tell about functional and usable aspect of the payment page in -- a mobile website and the payment web view in native/hybrid app

Benefits of this Approach

1. I and my tests will have clarity what is in my control and what not
2. When I have control, the test and automation can be well maintained
3. The flaky areas can be identified; I can come to a decision to eliminate it from the automation or not
4. It helps to identify what is my problem and what is the problem that I don't own in terms of authority
5. With this approach, the tests and automation provides clarity when we uncover a risk or problem

While I know the benefits, I must also know the cost of having this approach.