Showing posts with label Automation. Show all posts
Showing posts with label Automation. Show all posts

Friday, June 27, 2025

Give a Name for the Base64 Image in an Extent Report

 

I read the below question in Testing Mini Bytes community.

Hi All, I am using extents reports in my c# automation framework. I am trying to attach the screenshot but it is always showing with name as base64img. Any idea how to overwrite that base64img to customised text?


Capturing a Screenshot in the Test Run

With the Extent report, one can capture the image of screen during the test run.  This capture can be on certain condition or state being met.  That is, it can be -- on fail of a particular action; in a transition between the states; etc.

In an Extent report, when the image is captured as Base64 image and the thumbnail is set to false, the images captured will have name "base64img".  Clicking on this, I see the respective image.  


No Customized Text as Name for Base64 Image

Say, a Extent report has 10 images captured as Base64 image with thumbnail false.  All these 10 different images will have the same name -- base64img.

  • This can be more chaotic to interpret when a test run has captured more than one Base64 image, and, all have the same name -- base64img.
  • Do you see that? 
    • How do you know what is that image without opening it just by reading base64img?
    • That's hard and you have to make additional clicks to know what is in each image!
      • Is this a better use of time of you who is reading the Extent report?


Naming the Base64 Image in an Extent Report

I see, the semantics in C# should also be similar to what I'm sharing below.  




The above gist should help to have a customized text given for each captured Base64 image.

In this case, the captured Base64 image name will be the customized text "Resend_OTP_Button_Not_Available" and not base64img.  This makes it more contextual and relatable for the reader of an Extent report.



Friday, July 26, 2024

My First Hand Analysis of CrowdStrike Falcon Update Incident


I attempted to analyze the process dump of CrowdStrike shared by my friend.  He said, there could be an attack which is leading to crash of Windows OS globally.  This made me curious to look into the dump and learn.

I had no much context around it, but, a test engineer in me did not sit quite.  I started to analyze the dump information.  Here is my first hand analysis that I made on 19th July 2024 post 10:30 AM IST.


What I Saw?

  • It is a Windows OS's process dump.
  • Looks like something with C or C++ application reading how the memory offsets were in the dump.
  • It started to read a memory offset.
  • Then the process witnessed an exception.
    • Here the program could not read further
    • Why it could not read further from this offset?
      • My little experience of testing drivers on Windows OS for a card printer machine, refreshed and recalled what I had witnessed when testing.


Scratching and Striking My Mind


I started to ask these questions myself while I asked what could have gone wrong.  I could not stop here as I was curious what led Windows machine crash.  I referred to web and learn there was an update by CrowdStrike, and then this incident.

The bugs do exist in every software no matter the level and depth of testing, automation and engineering's excellence.  All software do crash and OS is not an exception to it.  But, what made the update to crash the Windows OS?  Pointing and blaming CrowdStrike or Microsoft is not a way for the practicing test engineer.  If these two organizations are serving its huge customer base, they have something working and reliable.  Engineering does not eliminate problems.

By now, I had a thought that it is not an attack.  It is a software bug!  Where is the bug?  What is the bug?  Was it not experienced in pipeline?


The Open Ended Questions


I had these questions as I analyzed and spoke to my friend.
  • What is Falcon?
  • What was this update to Falcon?
  • How frequently the updates are rolled out?
  • How the updates are rolled out globally?
  • What pipeline do they have in testing?
  • Who is impacted the most in business? Is it Microsoft or CrowdStrike?  Impacted in what way?
  • What is CrowdStrike?  What they do?  Who are the customers?
  • Where do the CrowdStrike's Falcon sit in the OS and what it does?
  • How CrowdStrike works in the machines and what it offers?
  • What do the dump say? Relook into it with different perspectives.
  • How this could have been prevented?
  • How will I prevent this if I join this team knowing this incident?
With these questions, I started to analyze the process dump which was shared.

I had more such questions, but these were the first few that I crossed as I started.



Analysis of Process Dump


My interpretation, tells me the below for today
  1. Accept that it is an incident as any other incident which I witness in production environment.
  2. Do not fall to the speculation happening around.  Remain calm and focus to interpret and understand your exploration.
  3. I see, if it can start to read from an offset and then ending to experience a non-existent or invalid offset, is it a NULL Pointer?
    • What is NULL Pointer?
      • A NULL Pointer is a pointer that does NOT point to any memory location and hence does not hold the address of any variables.
      • If I do not initialize and assign, the pointer will have NULL as its value.
      • For example, int *test;
        • When I want to access the pointer test (a location in memory) pointing to, I will not be sure what is in the pointer when I read it.
          • I may not set it later or set it.
          • In this case, the code can tell if the pointer is valid or pointing to a garbage memory
        • But, if I declare it like int *test = NULL;
          • I can check if was set and initialized
        • It is a better practice to assign a NULL value to a pointer during initialization so that we can check if it is NULL or as any address assigned to it.
      • This understanding of Pointer makes me think, is it due not initializing a pointer and so the error code c0000005 on reading a memory that is not valid.
      • When we assign a NULL value to pointer, it is a null pointer in C++
        • We assign null value for testing and asserting
          • If the memory is allocated to a pointer or not
          • If it has a return address and is a valid one or not
          • If a pointer is not initialized, assigning null it prevents problems to certain extent
    • With this understanding, I also read, it started to read from an offset 0x9c, and then failing.
      • What is 0x9c?
        • In Octal it is 234. In Decimal it is 156.
        • Can there be such address in a computer's memory? I don't know.
        • If it is a access violation, then is it a memory which is in preemption of the OS?
          • If so the OS can terminate the program or process which is trying to access it.
          • Is this killing the process and aborting the operation of Falcon's IPC and eventually Windows coming to BSOD?
      • This tells me it is not a NULL Pointer in first case but not initializing a pointer to NULL.
        • I infer, if the pointer was assigned to NULL, that is initialized, there could have been some hint in the state and event when accessing the memory.
          • This is my analysis; but, I have not seen the test code nor aware of the product.  All this inference is based on the process dump and my experience of testing drivers.
      • It got something in between from update (a config or pattern?) for which it cannot find and read in the memory?  Why?
        • This indicates me, it could be a bug, that is, a logical problem.  This is my hunch for today!
  4. Data in the dump
    • Exception Address
    • Read from Address 0x9c
    • Exception Code: c0000005 (Access violation)

Testing my Interpretations


CrowdStrike as an org when it caters its SAAS to such a customer base, won't it have a testing pipeline
  • It will have, I have no doubt in it.  They test and roll out the updates, I believe in it.

Did they witness any such incidents earlier?
  • I searched on web for it and I did not find something similar on the Windows, earlier.

Is this a NULL Pointer?  Are you sure?
  • No, I'm not sure.  But, there is something that is leading it to address which does not exist or which is invalid?  I will have to wait for their RCA to know technically what caused this.  But this is my understanding reading the dump.

How do you think it is a memory access problem?
  • The error code 0xc0000005 says that.
  • I referred to driver easy website for the information because my experience of testing the drivers for Windows OS and experiencing such incidents led me there.  This is what I learn:
    • https://www.drivereasy.com/knowledge/solved-how-to-fix-0xc0000005-error/

Do you think the programmer would not have handled the obvious Pointer and NULL initialization?
  • I believe there will be a check for Pointer and what it is pointing to.  But is it due to no initialization?  Technically this has to be analyzed which I cannot do.  I will have to wait for CrowdStrike team to share the tech details.

Is this a driver problem that killed the Windows kernel?
  • I don't know.  But, the .sys file will not have driver as per my learning.  It will have information about the drivers and any configurations.
  • This incident is a problem, which impacted both CrowdStrike and Microsoft.  Maybe, both will have their areas to look and fix it they see so.  But, in this context, CrowdStrike can fix it quicker and that is much better -- is what I understand.
  • I'm a Windows user for long time.  I see, Windows has worked well to all my contexts so far.  The Engineers of Windows OS knows better than me here.  I'm not well aware and informed as they are.
  • CrowdStrike's engineering team are skilled and they are rolling out updates often in a day.  They have a better pipeline when this is being done.
    • But, the question I have is, how did this happen?
    • No one lets such problem into production when they are aware of it.  Do you?
    • There is something that has not come to their observation and experience.  What is that?
    • Knowing this will help to prevent this and similar incidents happening in future.
      • I'm waiting to know what did not come to their experience and led to this incident.

What could be in the .sys file of CrowdStrike?
  • I don't know!  I want to learn that.
  • But, from my testing of .sys file and drivers on Windows OS, I learn there could be a configuration details with certain pattern or information to capture at run time, and help the installed software to run.  This is my learning and awareness from my testing.
  • That said, testing at OS level and Anti Virus engines are not obvious.  Testing of drivers is like the risky mines.  What is sufficient and good enough in test coverage?  It needs an expertise at OS internals level.
  • Windows OS having such a fragmentation in its versions, updates and patches, it is a battle field and mines for engineers building such solutions for sure!
  • I learn, the Windows OS stopped when an application tried to access the invalid region or non-existent memory.
    • The update which was rolled out, did it have a configuration or a pattern that showed a logical problem when processing it?
    • I have such questions and thoughts that are striking my mind as I think and build a problem model for the same.

Is this a race condition incident?
  • I see, it is not a race condition incident as users across globe experienced it.

Is this specific to a Falcon version, OS version and hardware?
  • Not all host machines would be on latest version of Falcon, is my presumption.
  • At least, n-1 and n-2 versions should be on host machine which experienced this behavior.
    • So it is not a Falcon version specific, I see.
  • It looks to me as it is not specific to the Windows OS version and hardware configuration.
    • It is an application software problem which occurred at driver level is what I see.
      • This is an IPC communication and process is my understanding.
        • The driver can receive the IPC communication in continuous mode.
        • At times, this can get queued based on the application and what it does.


Where is the Problem?


Well, I'm looking and pulling from my visualization by relating with my experience of testing the driver on Windows OS.  I don't know the exact reason or close enough to tell what could have gone wrong.

Reading the process dump, it says accessing a memory that does not exist or corrupted.  One of the high possibility is, the starting offset is seen but it is not helping when reading.
  • For example, Ravi has the address of India's Prime Minister house.
    • But, he does not know from where to start despite having the address.
    • He is void and null in knowing where to start and what to do when he is not initialized with the start location to begin the travel to the Prime Minister's house.
    • In short, he do not know where the address is pointing to and what it has, though he is given a address to start.
      • Can he access the Prime Minister's house premise without any access granted and authorized to do so?
      • If not, won't he be arrested by police or other security forces and stop him?

Do I Know the Precise Problem?


I don't know!  I do not know the CrowdStrike product and platform.  I'm waiting to read the technical details from Crowd Strike.

I see, it comes to the data, state and event.  I would focus on how to prevent it learning which data, state and event led to this behavior.  I think of figuring out the Test Design and Strategy that can help me to identify such use cases.  I focus here and see can it brought into the automation so that it gets exercised and regressed consistently.

If it is due to the memory access that had a problem, I did such tests when testing driver for a hardware machine on Windows OS.  I will share the tests that I did in upcoming blog.

I wrote the technical analysis from process dump to CrowdStrike and Microsoft.  I did not get a response.  Anyways, I'm sharing the overall information in a non-technical way so that it is consumable to most readers here.



Note: Here are another threads of me sharing my thoughts on same:
1. https://x.com/testingGarage/status/1814215089525821763?t=XSFdx69ElL0ZmBOcEFrTjg&s=19
2. https://www.linkedin.com/posts/ravisuriya_%3F%3F%3F%3F%3F%3F%3F-%3F%3F%3F%3F%3F%3F%3F%3F%3F%3F%3F-%3F-activity-7221156949445206017-oeRa




Friday, February 2, 2024

Deep Link and its Testing via Automation

 

I get these question consistently from my fellow testers and community.

  1. How to automate the mobile apps and web applications using Deep Links?
  2. How to automate the business flows using Deep Links?
  3. How to achieve end-to-end business flows testing on using Deep Links?
  4. How to automate scenarios in mobile apps using Deep Links?
  5. What is the best approach to automate the mobile apps using Deep Links?
  6. What is the best practice to automate using the Deep Links?
And, more questions on same pitch.


No Deep Dive into - What is Deep Link?


A hyperlink in HTML is a kind of deep link within a website or to another website.

Deep Link is known with different names for web, Android app and iOS app.  All these names have the same understanding and intent at some point.

The Deep Links are URIs that takes me directly to a specific part (activity or fragment) of the app that I'm using or testing.  The Deep Link will have an intent which tells where I will be taken on using it.

When we converse on diving deep technically into testing and automation of Deep Link, will share more insights into its internals.



Deep Link and Challenges


This question is discussed with me often:
How to do end-to-end testing using the Deep Link?
Automation of a mobile app using Deep Link poses a challenge which is not experienced in web application.  

One such challenge is, say you have not installed the mobile app.  [This is solvable!]
  • On using a Deep Link, I should be taken to Apple Store or Play Store based on the app.
  • I have to install the app.
    • Post this, in the traditional automation, I should start traversing the business work flows via GUI.
    • Is this adding to the flakiness aspect of automation via GUI?

When we talk so much about flakiness and how to avoid (not prevent), should we exercise business workflows when automating using Deep Link?  What you are thinking?  Let me know!



Scoping of Automation Using Deep Link


Back to the fundamentals.
  • We have to automate, no escape from it.  Let us automate what must be automated!
  • Let us not fall into trap of "Automate everything!"
    • For today, I'm in this mindset and attitude,
  • What we automate depends on the objective or goal that we want to accomplish.
    • Each test should have precise and deterministic goal.
      • A test via automation is not an exemption to it.
      • A test defined in automation should be precise, deterministic and have a single objective - Single Responsibility Principle.

What is the objective of my testing via automation for the Deep Link?  This define the scope and extent of my automation.  This will minimize the number of checks that I do using Deep Link.

The purpose of Deep Link is to take me to specific part of the mobile app.
  • Should I start the end-to-end or exercising the workflow to be included in the Deep Link tests?
    • If included, am I not complicating the testing via automation?



Automation using Deep Link

I ask this question to myself and to my team.
What is the goal of testing via automation using Deep Link?

This question helps me to pick minimal and necessity flow actions.   It has lead and leads me to define minimal tests for Deep Link based on what we want to learn from automation of same.

To me, the purpose of Deep Link is not end-to-end testing.  It's purpose is,

Am I taken to the intended state and data when used the Deep Link?

I have kept the test intent to this.

With this, I have come with tests that has minimal must evaluation and assertion to learn if the app is responding or not to the Deep Link.  This is what the business wants when the Deep Links are created.

The app usage and workflow function is not a problem statement of Deep Link in a general context.

Deep Link is not for end-to-end.  It is to take to you from a point to another point, that's it.


Are you automating using Deep Link?



Wednesday, August 16, 2023

When I Questioned My Learning -- What is Automation?



I'm questioning, revising, refining, and versioning to update my learning and practice with Automation in Software Testing & Engineering.  This blog post is about how do I define and understand the Automation and Software Testing.


Automatic and Ation

I started by breaking it into classes (words).  I see the better picture of what it does and what it means.

  • Automatic
    • which runs on its own with little human intervention where it is needed.  
  • Ation 
    • means an action
When put together, Automatic + Ation = Automation.

I do not want get into from which language did it come from.  I got what I need here.



Automation and Software Engineering


Today, the context, area, scope and subject of Software Engineering has grown when compared to last two decade.  This is a immense change, progress, and growth.  I can think of automation in each area of Software Engineering, today.

I will stick on to context of programming the code, testing of this code, and what to automate by learning how and why.  My work and practice is here for last one decade.

I unlearn and learn by redefining to myself:
Automation is the use of technology to execute a task with reduced human efforts.


I said, reduced human efforts and not elimination of human, and human efforts.
 



What is Software Testing?


It is a parallel engineering that is engineered in Software Engineering.  When said engineering, it is about civilization, learning the [actual] problem and solving the [priority] problem.

Then, Testing is about learning how well it is being engineered.  It is an exploration, study, execution, evaluation and connecting all these to know the differences if any with its value and cost.

Testing is part of Engineering without which  Engineering cannot exist.  Testing goes in parallel as an engineering to the engineering.

That said, Software Testing is an engineering that runs in parallel to Software Engineering.  To me, the definition of Dr. Cem Kaner for Software Testing looks to be closer as a engineering practice.  It says,
  • Software Testing is an
    • empirical
    • technical
    • investigation activity
    • carried out to provide quality related information
    • to stakeholders
    • so that they make informed decision

With the help of Software Testing, to an extent I will learn:
  • In what all ways the Software System appears to work - how, why & when
  • In what all ways the Software System fails - how, why & when
I will have a technical data that is rational and empirical to present my learning from the testing I do.



Automation in Software Testing


I'm learning consistently for over a decade now that automation in software testing is one of the ways to test and this has its own constraints and limitations.   It exists to assist the Software Testing & Engineering, and not to replace the Software Testing & Engineering.

When said automation, I use to think of Programming Language and executing a test via the code where the intent of the test is expressed.

But, today I learn, I can automate in multiple ways and in few context even without using any programming language.  Yet it is still an Automation in Software Testing.

I used a toy train to automate a use case of payment via card and NFC.  This worked very well in fact with actual card and POS equipment.

Most times, one equates Automation to the code.  It is not wrong!  But it does not have to be code of a programming language always, either. 

As I said, if Software Testing is an engineering, then, Automation in Software Testing is also an engineering.



What About You?


Did you redefine your understanding by unlearning?  Be it yes or no, I'm curious to listen you.  Let us catch up!


Tuesday, August 8, 2023

How To Start Learning The Automation?

 

I had this question.  I figured it out how to help myself.  I figure out better ways every day.  I read this question by other Test Engineers on forum and communities.

Hi everyone

I want to start learning automation testing.  Where to start and how start?  Can anyone guide please? 


You might have across the similar question.  And, you might as well have this question. Do you?

I share here in the way I say it to myself!


How Do I Start Learning Automation?

  • Find all the ways you can for how to fall in love with Programming
  • Fall in love with Programming
    • Choose a language to express your love
      • You have to express and practice better ways to express each day
        • You will get better in proposing as you express it each day
    • Is it one way love?
      • Don't worry! It will be two way soon
      • Let Programming understand you, and you understand the programming
        • Love at first sight is not for all
        • Some will take enough sights to fall in love
          • And after that, you know; if not, you will know
          • Do not give up! One small step at a time
  • See the Testing in your work
    • Improvise your Testing each day while you are in love with Programming
      • Testing will help in getting you and programming to love each other
        • Take help of this friend very well by understanding who is this friend
  • Learn consistently how to express the intent of your test via the code you program
    • Know the limit of the test intent you have expressed via the code that you have programmed

If you noticed there is no automation libraries, tools and Test Runners mentioned here.  All that is mentioned to get started is:
  • Programming and love for it
  • Understand and seek the help from friend -- Testing
  • Know the test intent expressed in your programming and its limitations

I want to do better and effective automation in testing to help myself and my testing.



Saturday, March 25, 2023

Black Box in Every Other Box of Software Testing

 

Modeling Software Testing With Boxes


The fact is something that is not put to scrutiny or questioned much and often.

As a fact, the Software Testing is explained to us using boxes.  That is,

  • Black Box
  • White Box
  • Grey Box

Is this wrong? No, it is not wrong.

There was a need to explain for one how to visualize -- how a person would interact and interpret the software system when testing.  The analogy of these boxes helped and helps.  These boxes are mindsets.  In a way, these boxes are like models to interpret the different ways and approaches we take in Software Testing.

I see, we are seeing Black Box in every other box.  Maybe, this is limiting one not to think to learn software testing in other than a black box mindset.

If you ask, are we not automating? Is not that a Grey Box?  Very much, we are a black box mindset when writing automation as well.  I include myself here.  I'm exploring how to break out of this and see the Software Testing.

Do programmers think of their code as different boxes?

  • A programmer reads, writes, deletes, and views the data. events and state
  • The programmer as well cannot see what's happening between the binaries on the electric circuits
  • The programmer evaluates her/his code's testing via logs, debugger, and assertion for the data, state, and events
    • Is this a white layer or box?
      • Is it called white because one can see through the logs, debugger, and assertion?
        • But that is still not a sight of binaries on the electric circuit, right?
        • If one could see through the binaries, we should not be having race conditions, out-of-memory, and unhandled exceptions
          • Isn't so?

This makes me feel, there is a Black Box in every box and we are largely confined to this Black Box.  

Exploring to step outside this box helps to understand the testing, software testing, system [and software] under test, and what is needed to test better.


Tuesday, March 14, 2023

Maven For Software Test Engineers - 101

 

Why 101s?

In the initial days of my career, I looked and referred to multiple sources to understand the technology stack.  As I progressed, I looked for someone who can help me in understanding what I'm reading and to clarify the questions I witnessed.  The understanding of concepts took time when I actually started using it and especially when debugging.

Today, if I look at that, I ask myself

  • How can I make my learning experience a happy one?
  • How can I capture the fundamentals and have a smooth revision of it when in need?
  • What can I have for my reference when I'm implementing the fundamentals?
  • How can I increase the speed in which I do all these?
If seen, I want to help myself to be better and effective when solving a problem by implementing what I'm learning.

Maybe, other software test engineers too have same question.  Don't you?

I'm learning new stuffs, and also unlearning what I'm learning.  I create the 101s for myself.  I see the software testing community can also make use of these 101s that I'm building.


Why You Need Maven 101?


  • Today, it is a common sight that one quickly picks up a boilerplate code and run it
  • Especially with the software test engineers, who wants to start/practice automation
    • Pick a framework and run a command
    • Execute the program successfully
      • Or, get stuck in between and do not know why is that command
      • Or, see an error with that command and not sure what that command is doing
      • What is the difference between these commands and what does it do?
    • As this, one can get into different contexts where one gets trapped; not blocked
      • How to help self to navigate out from here?
      • I see this 101s can come to help

Anytime, you picked a Maven project and built it?  For example, a Selenium code written as a Maven project.  Do we know what Maven is and why the Maven project here?  And, what these different Maven command means and what it does?  For having this understanding, this Maven 101 comes to your help.


Maven for Software Test Engineers 101


I have my Maven 101 here.  I refer to this source when I revise my fundamentals.  This can be of help to you as well and also can be a start point.

This has helped me to understand the different build life cycles and how a Maven command is all about the plugins.

I have provided the Reference section which is also a credit that I give them for helping me to unlearn and learn better.



Wednesday, February 15, 2023

When to Start the Automation in Software Testing?


The Question of a Decade?

Today, when I draft this post, the calendar date is 14th February 2023.  If I look back to 10 years ago and ask myself what are the questions in and around Software Testing and Automation, I see this question.  What is that question?

When to start the automation?

We answer, hear, read, and discuss this question, today too!

Often, the opinion that comes out is, ".... to automate when it is stable".  Note that it is an opinion, not an answer or a fact accepted universally.


To Start Automation when it is "Stable"!?

My learning is,

Do not think of starting the automation when it is "stable".

The "stable" is an assumption we tend to believe by the outcome of using the system.  The binaries are never "stable".

The binaries appear to not show any risks and problems for the way one is using it in a context.  To be more precise, we are not seeing the risks and problems that binaries are showing us in other dimensions.  That is, the dimensions that we are not aware of or the dimensions that we are not focusing on.


When to Start the Automation?

I learn,

Start the automation, when the system is testable!

This leads to me the questions:

  • What is testable?
  • When it is testable?
Understanding testability helps me to learn and identify its child attribute -- automatability.  That is, understanding testability helps me to learn the order of "testable".

Testability does not mean "stable".  Testable does not mean "stable".

But the assumption "stable" means there are some characteristics of testability, automatability, and order of testable.

Automate when you learn, it is testable, and identify a layer of testability.  This helps to pick the better seam [that is the appropriate layer(s)] for automation in a given context.

Keep the automation structure ready, so that intent of a test can be expressed via code as we identify [a layer of] testability.

Maybe, this is what people say LEFT or SHIFT LEFT or START LEFT.  Or, could be out of the SHIFT LEFT BOX!



Thursday, September 22, 2022

WebDriver: Tracing the Interface WebDriver - Part 2

 

In the previous post of this WebDriver series, I shared a gist about what WebDriver does and how.  In this blog post as Part 2 of this series, I'm sharing a bit more details on WebDriver and RemoteWebDriver.  

From there, we will see how AppiumDriver is related to WebDriver -- which extends the interface SearchContext.

This blog post is written as part of 21Days21Tips from The Test Chat.  The tip shared in this post is to know more about WebDriver internals and how it associates with RemoteWebDriver and AppiumDriver.

This should help in understanding the Selenium APIs better and from where it comes.  This helps in having a better mental model of the Selenium WebDriver and how we want to structure the instructions in the tests and utilities we write


SearchContext and WebDriver



Picture: Representation of SearchContext and hierarchy of WebDriver


  • The SearchContext is the parent interface in the WebDriver hierarchy
    • The subinterfaces of SearchContext are
      • WebDriver
      • WebElement
  • This SearchContext defines two methods
    • findElement(By by)
      • Modifier and Type is: WebElement
      • It finds the first WebElement using the given method
    • findElements(By by)
      • Modifier and Type is: java.util.List<WebElement>
      • It finds all elements within the current context using the given mechanism
        • NoteI'm referring to Java APIs of Selenium in this blog post
        • More details of this can be found here.

Note: Selenium's Ruby client describes the Interface SearchContext as this.  


The WebDriver provides the below methods:
  • close()
  • findElement(By by)
  • findElements(By by)
  • get(java.lang.String url)
  • getCurrentUrl()
  • getPageSource()
  • getTitle()
  • getWindowHandle()
  • getWindowHandles()
  • manage()
  • navigate()
  • quit()
  • switchTo()

More details of these methods can be found here.


RemoteWebDriver and AppiumDriver


Further, we see the class RemoteWebDriver implements the interface WebDriver.  Today, the WebDriver and RemoteWebDriver communicate using standard W3C specifications.

That way, all the modern browser which adheres to W3C specification should not have (much) trouble when using WebDriver and RemoteWebDriver to mimic the user action on them.  We see the ChromiumDriver(), ChromeDriver(), FirefoxDriver(), Edgedriver, SafariDriver(), and OperaDriver() extending the RemoteWebDriver.

This hints us to know and learn:
  1. Why do we initiate the WebDriver for first
  2. And, then we instantiate the browser's driver
  3. Later how we use WebDriver's instantiation to drive action (mimic the user action) on the browser using the respective browser's driver
When we want to automate using Selenium Grid, we make use of RemoteWebDriver to drive the action between the client and server.

The class AppiumDriver extends the WebElement and RemoteWebDriver from the project Selenium.  And further, it has its own methods to interact with the mobile elements.  More details about the Java Client of AppiumDriver can be found here.

The subclasses of AppiumDrivers are:

  • AndroidDriver
  • iOSDriver
  • WindowsDriver


21 Days 21 Tips -- #day17

Here are my pointers to fellow test engineers

  1. Interface SearchContext is top in the hierarchy of the WebDriver interface
  2. Interface SearchContext defines
    • Should I want to search for the element in the whole page
      • using WebDriver object
    • Or, should I search within a containing element
      • using WebElement object
        • We can notice methods returning the type WebElement
  3. RemoteWebDriver implements the interface WebDriver
  4. The modern browsers drivers extends the class RemoteWebDriver
  5. AppiumDriver extends the class RemoteWebDriver and interface WebElement
For more understanding of the SearchContext and WebDriver, refer to below git repository of SeleniumHQ:

The below understanding should give a mental model of how the call happens in Selenium's library:
  • WebDriver and browser's driver instantiation
  • The order in which it is instantiated and used in programming to automate actions on the  browser

If noticed, the automation we do is more of programming and not of Selenium's library.  We extend and implement the Selenium library in our programming to mimic the action on the browsers and mobile apps.


Monday, September 12, 2022

Testability: More About it from the Programming Literature

 

My friend Parimala Hariprasad gifted me the book Essential Skills for The Agile Developer, authored by Alan Shalloway, Scott Bain, Ken Pugh, and Amir Kolsky.  Thank you, Parimala, for gifting this book.  I'm experiencing the value of this book and using it.

In this post, I'm sharing the content shared in Chapter 3 of this book. It is about Testability and how it improves the code quality.  


Why this Blog Post?

I continue to read Software Testing literature.  I understand the below as one of the primary key skills for a Software Test Engineer practice:

  • Identifying the Testability attribute in the system
  • Mapping and classifying how the available Testability attribute can be used in Tests
  • Asking for the Testability attribute
With that, I understand "how easy it is to test by a test engineer in a given context" as Testability.  If noticed, this is from Software Testing literature.  And, I see it has these three elements which tell the prominence of each:
  • How easy it is to test?
    • what factors make it easy to test?
    • how does it make it easier?
    • how does it bring the deterministic character?
    • how can I isolate the observations with my analysis with the help of deterministic character and aid added?
  • By a Test Engineer's
    • awareness, experience, learning, applying the skills, and more
  • In a given context
    • time, people, environment, availability, and more
If any of these three elements has trouble, it has its effects on the test and testing.  If you ask what effects, I don't know.  If I pick from my case to share one of the effects, I say, I was not very sure what was happening though the product looked to do what is expected.  But will it continue to do what is expected to do and in what all ways? I had no answer for in what all ways and in what contexts. This is one such case of how the absence or not using the Testability can influence the tester to be unsure about the learning made with help of a test.

The book I mentioned here gives another perspective from the Computer Programming literature.  It talks at the fundamental level and I see this is important to understand for we Test Engineers.  Soon in the coming days, we Test Engineers will be working and testing in these layers of product development. 

In the next section, I will share the lines from the book as is in italics and blue font color word.  The credits are to the authors of this book. I'm taking the text as it is from this book.  And, I will share my interpretation for the same and see the relativity of Computer Programming and Software Testing literature.  

Note: The credit is to James Bach for the Testability definition used above.  I added "the tester and context" to it as these two influence the Testability and outcome of using the Testability to a greater extent.


Testability and Code Quality

The authors of the book say, "testability is highly correlated to the code qualities we want to manifest, in particular, loose coupling, strong cohesion, and no redundancy."  Further, they illustrate how one remarks at the start of testing one's code by saying the below:

I can't test this code; it does too many things that are so interwined -- weak cohesion

I can't test this code without access to dozens of other things -- excessive cohesion

I can't test this code; it's been copied all over the place, and my tests will have to be duplicated over and over again -- redundancy

I can't test this code; there are too many ways for external objects to change its internal state -- lack of encapsulation


Then I read this line from the authors, "Gee, I wish they had thought of how this code was going to be tested while they were writing it!".  That's a question that every one of us has to ask ourselves for the work we deliver and not just for the programming.  

Alan Shalloway says he is kind of slow sometimes because it took him some time to realize this -- I should consider how my code is going to be tested before writing it! 

Testability is related to loose coupling, strong cohesion, no redundancy, and proper encapsulation.  Another way to say this is:

  • the tighter your coupling, the weaker your cohesion; 
  • the more your redundancy and the weaker your encapsulation, the harder it will be to test your code
Therefore, making your code easier to test will result in the loose coupling, strong cohesion, less redundancy, and better encapsulation.  This leads to a new principle -- Considering how to test your code before you write it is a kind of design.

Since testability results in so many good code qualities and since it is done before you write your code, it is a very highly leveraged action.  That is, a little work goes a long way; it is a great trim tab.


I and Testability


I try to understand and learn about Testability every day in my practice.  When I started my career 15 years back, I learned from my network, that one of our fellow testers in the community that is Meeta Prakash did her Ph.D. in Testability.  I wanted and still want to read the thesis of Meeta Prakash.  I hope she will find it and give me soon, one day.  In those days, I referred to the slides of James Bach on RST; that legacy slides that had contents filled with blue color. 

From there, I tried looking into the testability in what I test and what programmers deliver to me.  When I worked with Moolya in 2012, I realized from my practice -- context and the skill sets of a tester matter to make use of the available testability and to identify if it is present or not, and to what extent. I added this to the definition of James Bach and I shared the same with my fellow testers with whom I was mentoring and working together.


Relating the Literature and Interpretation


When the programming is talking about testability, I see it is talking about:
  • internal aspects of how it is programmed and to test the same easily in isolation, and in integration for the context while being deterministic
The words used to express in programming literature are more programming oriented.  Whereas, what we see in the Software Testing literature, it is more of a common man's words.  But, what both means is the same and the difference between them is to which layer and aspect they are referring and how, and why.

The Weak Cohesion
  • It will be an obvious experience to a tester when it is difficult to speculate and pull a particular observation with more information for a feature or a user flow
    • For example, if the Refresh Token is used along with Auth Token everywhere, then it will be tough to isolate when Refresh Token is used and when the Auth Token is used
I feel the same when wanting to test a piece of code in isolation from other code.  I have experienced this when testing one aspect of utility or a complete utility in isolation from the rest of the automation code.


The Excessive Cohesion
  • I could not test the mobile apps as I needed data
  • Certain data came from a portal that is also under development and depends on APIs to work
  • APIs would be under development till the last day of release and did not deliver the endpoints to the portal and mobile apps team
So how could the test team create data to test for mobile apps, web portal, and for APIs themselves?  If you see this is excessive cohesion at the product development level.


The Redundancy
  • In one project, I had to login each time to see the status of a session
    • All tests were programmed in a way that I should login each time
  • The test team used the login function in every test and it was duplicated
  • When signed in, the Auth token got changed which lead to difficulty in debugging and isolating the problem
This complicated the test code and also messed up debugging.  The tests could not be deterministic here.

I see a static Auth token or one-time login and using the same Auth token in all other tests in the suite could have helped to debug the problem and where it occurred.


The Lack of Encapsulation
  • My team had a tough time when started to use an existing automation
  • It had public access modifier for all methods in all packages
    • The team picked up and authored more tests that changed the data and states
  • This led to any object of a method to modify the data or state; it was not supposed to be modified at all
  • The debugging led us here and it was not a problem with the product
    • It was the problem with the product's automation code and how the tests changed data and state;  it was in turn used in other tests
This led to much more chaos as the automation and testing environment were the same.  The invalid bugs, meetings that got scheduled to discuss and time went into the meeting that ended with no use, and a couple of releases came into a decision should we deploy or not, and more.



Continuing the Unlearning and Learning of Testability


If you see, Testability has got multi-dimensions in the dynamics of software development.  Testability is not just about Programming and Testing.  It can be from the environment, project, people, what we understand and how we use it further in work, and the business itself.

I continue to unlearn and learn testability every day as I practice testing and automation. 



Wednesday, July 13, 2022

IntelliJ and Cache: Maven Dependencies Not Resolved


This post is about the Maven dependencies not resolving.  I'm recording the incident, my understanding, and what worked for me. It can help a fellow Software Test Engineer.


Incident and its Details

I'm using the machine which has the below setup:

  • OS: Windows 10 Pro
  • IDE: IntelliJ IDEA 2022.13 (Community Edition)
  • JDK: 1.8
  • Maven: 3.5.4 

I created a new Maven project and in the pom.xml, I added the below dependencies.  The IDE showed that these dependencies are not resolved.

<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>4.3.0</version>
</dependency>

<!-- https://mvnrepository.com/artifact/org.testng/testng -->
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>7.3.0</version>
<scope>test</scope>
</dependency>

<!-- https://mvnrepository.com/artifact/io.github.bonigarcia/webdrivermanager -->
<dependency>
<groupId>io.github.bonigarcia</groupId>
<artifactId>webdrivermanager</artifactId>
<version>4.3.1</version>
</dependency>

<!-- https://mvnrepository.com/artifact/io.rest-assured/rest-assured -->
<dependency>
<groupId>io.rest-assured</groupId>
<artifactId>rest-assured</artifactId>
<version>4.4.0</version>
<scope>test</scope>
</dependency>


I observed, that the added dependencies not getting resolved, that is, not getting added to libraries.  In the IDE, I see the error as this:

  • Dependency 'io.github.bonigarcia:webdrivermanager:4.3.1' not found
  • Dependency 'org.seleniumhq.selenium:selenium-java:4.3.0' not found 
  • Dependency 'org.testng:testng:7.3.0' not found
  • Dependency 'io.rest-assured:rest-assured:4.4.0' not found

Note: I have other Maven projects to which I have added the same dependencies but of different versions, in my workspace. Each time I add a dependency it will download to the libraries in my workspace though it exists, and this is my configuration in IDE.

At this point, I did not know what is happening and how to approach resolving this problem.  All I know is, that there are other versions of the same dependencies in the Maven's library on my machine.


Understanding the Problem


I tried to understand what it is saying to me.  It says it cannot find the dependency.  Could be that it is looking for the dependency in the Maven's library on my box, and it is not finding it.  

Further, I looked into the web to see did someone else face the same incident and said behavior.  Few posts said using the "compile" scope solved the problem in their case for WebDriverManager.  But, how can the change in scope fix the dependency not found incident?  This was the question in me!  To my curiosity, I did try that for WebDriverManager; I did not see any change.


Invalidating the Cache - Is that a Fix?


In the IntelliJ IDE, I tried to invalidate the cache and reboot the IDE and open the project.  I see the dependencies being resolved and I do not see the said problem.

I don't know how this fixed the said problem here.  Also, I don't know precisely what exactly the problem is, here.  Invalidating the cache and reopening the IDE (also the project) worked.

On invalidating cache, here is what happens per IntelliJ:
  • Removes the cache files for all projects ever run in the current version of IDE
  • The files will be recreated the next time you open these projects

I remember now that I upgraded the IntelliJ to the latest version before creating the said new Maven project, here.  Later, it resulted in the above said dependency problem. 

Note that, I had created a few Maven projects using an older version of IntelliJ.  I have the same dependencies in the old and new projects, just the version of these dependencies is the only difference.

Now, invalidating the cache on this new version of IntelliJ, it got fixed.  I understand that the IDE uses the cached instances of these dependencies.  Could be there is a relationship between this cache, dependency versions, and the IDE version?  I'm technically not sure of the same, but my instincts say it is.



Friday, April 1, 2022

WebDriver: Clarifying the Confusion on Why and What is the WebDriver - Part 1

 

I had a question "What is WebDriver and why should I use it to automate on a browser?"  I tried to understand it and relate its presence in code written using Selenium.  I see this question in the test engineers who are starting the practice of automation on browsers. 

And, most of us get confused with WebDriver, WebDriverManager, and WebdriverIO.  All of these are not the same but all these work around the same space that is automation on the web and mobile.

Between, I learn understanding of WebDriver is fundamental to the practice of automation on web browsers. The same idea is taken to the automation of mobile apps using Appium. 

I'm sharing this learning of me as a part of 21Days21Tips the initiative from The Test Chat community.  The tip here is to assist by providing clarity around the WebDriver and why we use it in automation on a browser.


What is WebDriver?

The WebDriver is part of the Selenium library and we use it every time when we are trying to do any interaction with and upon a browser.  It is also a language binding and helps to write the browser controlling code.  For example, if I pick Selenium's Java WebDriver,

  • it provides the APIs that I consume to control the actions on the web page displayed on a browser
  • likewise, if I pick Selenium's Python WebDriver it provides me the APIs that I consume to automate my actions on a browser
    • I code here using Python
That said, the WebDriver is a set of APIs and to be precise it is an object-oriented API adhering to the W3C standards.  As a result, the WebDriver drives the browsers effectively today as all popular browsers to the W3C standards.  The HTTP is used as the transport protocol.


Understanding the WebDriver

On a higher level, this is what WebDrier does:

  1. The tests we write make use of WebDriver API 
  2. This WebDriver API carries the commands (written in the test) to interact with the browser's driver
  3. On receiving the commands, the browser's driver and the browser will have native communication, where the driver will translate the commands to the browser to emulate the action on a browser.
  4. The browser returns the response to its driver
  5. The browser's driver will transfer information to the WebDriver
  6. Then, WebDriver shows the information to a user who is running the test
Examples of browser's driver are:
  • chromedriver of Chrome
  • geckodriver of Firefox


Representation of Selenium WebDriver's Communication


The instructions (commands) that I pass via WebDriver's object are translated to stateless information.  That is, there is no state maintained between the client and the browser's driver.



Representation of Selenium's WebDriver SPI & Browser Interaction

When the code enters into Stateless Programming Interface (SPI), it is called into a process that breaks down what the element is, by using the unique identification and then calling the command.  For example, let us look into the below statements to understand what the code looks like at SPI:


Code written using WebDriver API:

WebElement greetBox = driver.findElement(By.id("greeting_textbox"));
greetBox.sendKeys("Welcome to Testing Garage's Blog");

 SPI:

findElement(using="id", value="greeting_textbox")
sendKeys(element="greetBox", value="Welcome to Testing Garage's Blog");


Note: The findElement and sendKeys are the commands provided by Selenium's WebDriver API to find the web element on the web page and enter the text into the web element. The browser's driver receives these commands and data, then emulates the command (a user action) on the browser, and carries back the response to WebDriver.


21 Days 21 Tips -- #day13

Here are my pointers to fellow test engineers who are confused about WebDriver
  1. WebDriverManager and WebdriverIO are not WebDriver
    • But all of these are around automation of the web and mobile
  2. WebDriver interface helps in
    1. Control of the browser
    2. Identification and selection of web elements on the web page
    3. Provides assistance to debug
  3. Browser Level API
    1. driver.manage().window().maximize();
    2. driver.get("https://testingGarage.blogspot.com");
    3. driver.navigate().back();
    4. driver.navigate().forward();
    5. driver.getWindowHandle();
    6. driver.getWindowHandles();
  4. Few Page Level API
    1. driver.findElement(By by)
    2. driver.findElements(By by)
    3. driver.getCurrentURL();
    4. driver.getTitle();
    5. driver.getPageSource();
  5. If you notice, we use these APIs to automate the browser
    • The tests we write use these APIs of Selenium WebDriver along with the assertion
  6. Why are we using "driver" in the above commands?
    • This is another question and confusion among fellow test engineers starting to practice automation
    • I will share this in the next tip :)
This understanding of WebDriver, and the why and how it is instantiated (in the next post) will help you to be comfortable in starting to read the test code written using Selenium.



Thursday, March 10, 2022

Understanding - Are Framework and Library the Same?


We Have This Question

  • What is a Framework?
  • What is a Library?  

I had these questions when I started my testing career.  At times, I ask this question today as well, when I listen to fellow engineers talk about it.  

I can say firmly that you had these questions.  You would have asked these questions to yourself or to your fellow testers.


Using Interchangeably

In the teams in which I have worked and in the community, I have seen these two terms are used interchangeably.  This has added to the confusion.  At times you might see the programmers using these two words interchangeably; but, they might know what it does and how to use it.

If I were to start my automation practice, to what should I call framework and library?  Does this question have crossed your mind and thoughts?


Framework and Library - What it is?

I understand both Framework and Library as code.  Yes, both are programming code that is structured and organized in a way.  This code is written by a programmer and will be consumed by other programmers and testers.  

Why do we use it?  We use it to assist in solving a problem we have in our work.  May be is what leads one to get confused or use these words interchangeably.  But are they one and the same?  I had this question.  I spoke to programmers and testers with whom I worked.  I observe each of them having their own ideas and perspectives about it.  

But are they one and the same?  As of today, I understand Framework and Library are not the same. It differs in -- the ease of consumption as is and the code to be written and organized for using it.  I see these two factors differentiate Framework and Library; there could be more factors and I'm not aware of it.


Framework and Library - My Experiences


I use libraries in testing and automation I do.  For example, 
  • the Selenium library to interact with the browser and perform the actions on the web pages
  • the RestAssured library to build the HTTP request
  • the Appium library to interact with the mobile OS and its app's UI

I shared the above examples as most of us are aware of them and relate to them in our work.  Then, what is the framework?  

Have we heard of these words -- React, Angular, Bootstrap, and Flutter?  Today we use them to build web apps and mobile apps (using Flutter).  These are frameworks consumed by programmers to build the applications.  Some of these frameworks provide libraries to test as well.  For example, Chai and Jest assertion libraries.

Trying to understand further, one of my programming friends said me about the principle -- Inversion of Control.  Inversion of Control allows a framework to inject dependencies, inject configuration and manage lifecycle. That way programmer's program can focus on what it must do and not have to worry about any additional responsibilities.

I learn, Inversion of Control refers to transferring the control of objects and their dependencies from the main program to a container or framework.  How this will be implemented depends on the programmer and the design used to implement the same.

Note: In one of the perspectives I learn the framework provides the abstraction to its libraries.  We use these abstractions of a framework and develop our applications.  Can I say framework means abstraction?



Difference: Framework and Library


In the way we communicate, most times it is communicated as there is no difference between them and both mean the same.  Probably this is the reason we have the confusion, especially the Test Engineers who want to start the practice of automation will have this confusion.  This is being passed on to the next generation of Test Engineers who come take up Software Testing as a career.

Technically, the difference exists between the Framework and Library.  One of the key differences lies in the principle mentioned above -- Inversion of Control.  Let me walk through this with a use case that we Test Engineers cross every day.


Use Case in using a Library: Automating a user action on a web page

  • I make use of the Selenium library
  • I initialize the WebDriver and a driver of a browser
  • I open a browser on my machine using WebDriver communicating with the browser's driver
  • I talk to the browser to launch a URL
  • I make actions on the web page

I'm not writing a code to do all of the above said.  I'm making use of APIs exposed by the Selenium library to do all of these actions.  The library's code handles all these for me.  My code will be for -- to consume Selenium's API and to aid evaluation of business logic -- the Utils is a commonly used word to describe the Class which has methods for the same.  

When I'm using a library, I have control over the code that I write in automation in deciding how the automation should execute that is in what order and what flow.  I can decide when to make what actions on the web page here.  The Selenium library does not mandate it on me.  

The only mandate I see is the sequence of actions that has to be made when using a particular API.  For example, the action events to be made in what order when used the Touch Actions API.


Use Case in using a Framework: Building a web and mobile app

  • The team developing Android and iOS mobile apps using Flutter
  • The team developing a web using Flutter or React

Just by using the Flutter and React frameworks, the apps and website will not be ready.  The programmer follows the structure and sequence insisted by the framework in her/his code.  The framework will tell how the code should be written and probably this also means the flow of code is controlled by Framework.  For example, when loading the content on the web page and screen.

In one of my current projects, the place where the message is displayed, how it is displayed, and how to interact with it, is a flow in code.  The programmer has to work oblige how the framework parses the error message.  It has to be done that way unless the programmer writes a custom code and plugs it into the framework, provided the framework supports it.  

This code which the programmer plugs in now, it is a custom library developed to use with a being used framework.  Now using this custom-built library, the reading or/and display of error messages can be made as per the need of the product.



Automation: Do We Write Library or Framework?


When we testers say automation framework, I see that we are referring to the structured library we have written consuming a framework and library.  For example, Arjuna developed by Rahul Verma is a framework.  I will consume Arjuna and build a library that has tests to automate web applications using the library as Selenium.

The automation I write consuming a framework and/or a library will have methods. Few of these methods will be written once and can be used in multiple places.  That is no repeating code and practicing the reuse of code.  It is like DRY -- Do Not Repeat Yourself.  We can see this practice in the automation we write.

For example, the below code can be written once in a class and invoked in different places where we test the business logic.  Now, these two methods will act as a library code and it is also part of a library code.  




We just call the method name and pass the parameters in different classes where it is needed.  No need to implement these methods each time in business logic where it is needed.  

Doesn't this give a perspective that a framework includes a library within it?  And, this questions me --  should I call automation what I do as an Automation Framework?  I say NO!  Unless someone uses my work as a foundation to build their tests and it dictates how the code should be written and structured, I do not call it a framework.  

But, one can use the classes that I write in their work.  This class act as a library.  Recall the class file name with Utils; this file has methods that we use by invoking it across the package.  One good example of this is, the logging part we write in our automation.  This logging part of code can be built as a library and can be used in different automation projects in which we work.

This is my understanding of Library and Framework at this point in time.  You have any questions or other perspectives that I'm not seeing, do share.