Workshop Experience: Web Application Security Testing

 

This writing is my experience report of the workshop Web Application Security Testing conducted by Verity Software with Rahul Verma as a trainer. I had attended this workshop in November 2011. I registered for it again and was part of the workshop for the second time in September 2020. I thank Vinay Baid and Anil Nahata of Verity Software for coordinating and helping me to attend this workshop.

I and Web Testing for Security

I started testing for web applications in 2008 by learning what the browser is, its internals, and by understanding the web technologies. While doing this, I was working on projects which built web systems for -- SalesForce, Healthcare - Insurance, Reporting, and BI Reporting systems.  

One of the projects was supporting only for IE (IE6 & 7). The other projects web was supposed to support desktop Firefox, Chrome, and Safari. In the project, my task was to test for functionality.

In parallel, I picked testing for the security of these web applications. I referred to OWASP and its contents. I was building my mindset for security testing; I tested for web application security. I found the security bugs!

Disclaimer

I'm not paid by anyone to write this post and no one asked me to write one. I'm writing it has my workshop experience and learning I made out of it. I'm writing it to document my experience from this workshop.

About the workshop

I saw the post in 2011 from Rahul Verma about his workshop -- Web Application Security Testing, two days workshop. I registered for it in 2011; it helped me. In 2020 September, I did attend the same two days workshop from Rahul Verma and conducted by Verity Software this time. The detail of this hands-on workshop is available in the Verity Software's website.

  

I did not feel it is a repetition. In these eight years, my thought process has changed. I see that I have progressed in my learning in these eight years. Yet, I did not experience it as a repetition. 

The only differences that I could see are:

• In 2011, it was in a hotel at Kormangala; 2020's workshop was online as an effect of the COVID19 pandemic.
• In the physical workshop, the trainer moved around each table and looked into the trainee's practicing; in an online format, he helped by asking how we are doing the hands-on exercises.
• In the physical workshop then, the trainer had given a laptop if needed with software that needs to be used and installed; in an online format, we trainees had our laptop and seated at home, and accessed the practice system hosted on a cloud via remote desktop.
• In the physical workshop, he could see us, our eyes and our face, and understand what's happening with us; in an online format, he had turned on his camera, and trainees had turned off their camera.
• In the physical workshop, there was networking between the trainees; in an online format, no networking, and sharing between the trainees.
• In the physical workshop, he wrote on whiteboard and in a projected text editor and explained; in an online format, he used the Sublime editor to write and explained his thoughts on the shared screen.
• In the physical workshop, there was silence, we trainees listened to him; in an online format, at times we had trainees microphone turned on and could hear the background sound (I don't call it as noise!).
• In the physical, we did not see any break or lag in the trainer's voice; in an online format, we could see the lag in the trainer's and trainee's video and voice (latency, bandwidth, streaming & internet!).

Otherwise, I made my notes as I listened to him then and today. It was the first online workshop for the trainer Rahul Verma.

Why did I attend this workshop?

Here are my reasons why I attended this workshop:

• To check on my fundamentals, thought process, and mindset in Web Application Security Testing.
• To see the difference in me and my practice after I attended the previous workshop.
• To learn certain concepts better from a practitioner who practices web application security testing.
• To listen to Rahul Verma:
• He doesn't do sugar coat.
• Says what he knows and what he practices.
• His way of explanations and the way he looks at the fundamentals before security.
• His experiences and what kind of security information he finds and how.
• I did not connect to it well in 2011 as I connected to it today.  I was grasping slowly and thinking about what I do as Rahul Verma spoke.  I did not repeat this mistake in the 2020's workshop.
• Today, I received it better as I'm practicing it, and I could relate my work when he discussed subjects and topics in the workshop.

What I made out of this workshop?

I said to myself to unlearn and not to think with what I know as I listened to the trainer. I went with an open and listening mind to this workshop. I did make sure to keep myself attentive in the workshop. I share a few of my learning here:

• My fundamentals got revisited; registered it better in my thought and mind.
• Understanding and the way I see what I see is with more clarity and observations.
• The topics which look buzzy and complicated have become much simpler now to understand and work on it.
• My mindset is realigning with the unlearning I had in the workshop.
• I wanted to re-arrange my thinking here if I had to, and I did it listening to Rahul Verma for the second time.
• Before learning security testing, the fundamentals of the web were taken seriously and discussed it.
• I cannot write in detail about it here. Probably if I do that, it may impact the trainer and organization conducting the workshop.
• The fundamentals he discusses here are needed -- stepping stones.
• As we know, tools assist in testing better, but it does not test on behalf of a tester. Yet using the tool in security testing is helpful in context up to a limit, and later it is human who has to test for security. 
• I did not see anything I listened to as a repeat for me.

I got what I wanted from this workshop. It is on me now how I practice and lead myself ahead.

My experience and learning

What the trainer spoke is available in books and on the web. What's not available is the thought process and how to approach it by understanding. The demonstration of a practitioner has to be experienced in person if possible; it brings a different and unique value in the trainee. My peace is paced well and tuned. 

The value added by the security testing of a Test Engineer/SDET and Security Testing specialist is unique and needed. My idea of encouraging and assisting the Test Engineer/SDET to practice Security Testing is much strong and clear now. I will continue to practice Security Testing as a Test Engineer/SDET, and sure I will add my unique values early in the work I do.

I have got the confidence now that if I attend it another time, I won't experience it as repeated to me.  It will be new and unique.

If you can afford and attend this workshop from Rahul Verma, do attend.  It will help to build the fundamentals and mindset needed for the Security Testing and Web Application Security Testing.

Workshop Experience: Certified Selenium Engineer

 

This writing is my experience report of workshop Certified Selenium Engineer conducted by Verity Software and Rahul Verma as a trainer.  I could not attend the scheduled workshop, once.  Vinay Baid from Verify Software allowed me to attend the workshop on another date.  I thank him and Verity Software for the help.  In this, conversation, Anil Nahata helped me by coordination and I thank him as well.

Disclaimer

I have not been paid or asked to write this blog post.  I have decided to share my experience and take away from the workshop which I have attended.  I'm doing it, to have my learning experience recorded in brief.  As well, it can help the trainer in a way that her/his workshop will reach more people.

About the workshop

This workshop is on Selenium having the title "Certified Selenium Engineer".  More information about the workshop can be found on the website of Verity Software.  Per my experience, the Verity team is approachable and they coordinate learning the context of workshop attendee.

It was a three days workshop and it was conducted in a hotel at Koramangala, Bengaluru.  Following the workshop, there was a mock test and then a certification test on the third day.  The questions needed thinking to solve and to analyze and choose an answer  Not all questions were straight.

I did not feel the paper as tough.  It engaged me well and made me think.  On understanding the fundamentals well and with the practice that one does in the workshop, the test paper can be easily attempted.

Why did I attend the workshop?

Here are my reasons why I attended this workshop

• Rahul Verma
• I see Rahul Verma as a serious and thoughtful practitioner in the Software Testing space
• Listening to what he has to share in the workshop
• I know he will not just talk in and around Selenium topics that were mentioned in the workshop details
• What can I unlearn and learn listening to him?
• There will more and beyond Selenium, in using Selenium better for sure
• I needed this; I wanted to listen to this; I wanted to see how he thinks and interprets
• I said myself do not be biased with words and thoughts of Rahul Verma
• Think about what he says
• Interpret on what he says
• Understand what he says
• Think what you (Ravisuriya) think
• I see whatever he spoke in the workshop, it makes very senseful

What I made out of this workshop?

I got what I wanted from the workshop.  I made sure, I make most of it and I was attentive to it.  I share a few of them here:

• I said myself come with an open mind; forget whatever you know for a while; unlearn and look from a fresh mind
• I did not open my mouth to talk except for asking questions; I had kept my mind open and attentive
• I did not like missing any word spoken in the tutorial be it from trainer and attendees
• Rahul Verma shared beyond Selenium's basics
• Rahul Verma spoke about Programming in fact and how to use programming for making the best out of a Selenium library
• Starting with
• What is the web?
• How the web works?
• What the web page is?
• What is DOM, HTML, CSS?
• What is an element, tag, attribute, and value in the HTML?
• What constitutes the browser?
• What is Selenium?
• The architecture of Selenium and JSON Wire Protocol
• Design Patterns and Design Pattern used in Selenium library
• What is said in the syllabus were covered with examples
• Not just examples of automation using Selenium
• But also with an example of what to do and not do - this is a treasure!
• This is what I want to listen from the practitioner's experience
• Framework and Test Authoring
• If I did write everything in detail and how the topics were discussed, it might look like an exaggeration; but it is no exaggeration
• It has to be experienced at least once from the practitioner's training

My experience and learning

Whatever Rahul Verma spoke it is available in books and web content that talks about Selenium.  The difference is

• Concepts were said in simple terms
• Examples shared for the topic spoken were from his work experience
• Programming - do's and don'ts with each example and concepts
• Keeping it simple, workable, and learnable
• If want to be strong in the fundamentals of Selenium, this workshop helps

For the kind of my mindset of learning style and expectation, this workshop from the trainer gave me satisfaction.  I'm happy!

Not a Senseful Question: Is API Testing important than UI Testing?

It is usual for a tester/SDET to come across the question and discussion around API Testing versus UI Testing, asking which is better or more important. More questions which one might cross on the same topic are as below:

• Which testing helps the tester to find more bugs and faster -- API Testing or UI Testing? 
• Which testing is quicker and effective -- API Testing or UI Testing?
• Which testing is less flaky -- API Testing or UI Testing?
• Which testing helps to find risks and problems early in the development cycle -- API Testing or UI Testing?
• Which testing has less dependency and maintenance comparatively -- API Testing or UI Testing?
• Which testing provides quicker feedback on the health of product -- API Testing or UI Testing?
• Which testing needs little efforts and investment to build, execute and maintain -- API Testing or UI Testing?
• A tester can start testing with API before the UI is available, so which testing is effective -- API Testing or UI Testing?
• If API Testing is covered no need for UI Testing?
• Which tests cases while programming is in progress or yet to start -- API Testing or UI Testing?
• The APIs can be used to test web application service logic, so which testing is effective -- API Testing or UI Testing?
• UI affects the UX, so UI Testing is required; don't the API affect the UX? If both affect the UX, which testing is of priority and more influential?
• The testing of a system for data using API is possible before the UI is available, so which testing is effective -- API Testing or UI Testing?
• One can say it is functional just by using API with no UI, so which testing is effective -- API Testing or UI Testing?
• API Testing can reveal more details of services and web application server, which UI testing cannot with or without load balancers, so which testing is effective -- API Testing or UI Testing?

I hope that was not a big list of questions, and this list keeps growing if I share what I have come across as part of the reasoning. Probably, you might as well have witnessed such questions when discussing the API Testing over UI Testing.

The word "Testing" is vague!

I learn the word "Testing" is vague when used with no precise context and expectation. For example, "API Testing" does tell that subject of discussion and work is around API in one's testing. Also, it does not communicate precisely -- what in the testing of APIs. The same is for the testing of UI.

In most of our discussion, we see such words loosely said in conversation. As a result, it may lead to ineffective thoughts. For example, "Is API Testing important than UI Testing?". To me, this question does not tell -- What to provide as a response, so one can think about it and decide? On what basis the comparison is made for the effectiveness and in which context?

We can test API and UI for quality criteria as -- Functional, Performance, UX, Security, Accessibility, Simplicity, Usability, and more. Yes, we can test the APIs for UX.

How can one reduce the vagueness when the words become too casual?  

When used the word "testing" providing the context of expectation, it can reduce the vagueness in communication. For example, "Testing for the API functionality is important than the functionality of UI?" Now you see the difference and clarity with this question in comparison to -- "Is API Testing important than UI Testing?"

API and UI are individual operating entities in a System

To benefit the reader here, let me take Mercedes-Benz's car assembly line illustration. What can relate to APIs and UI here? Can the exterior of the car be seen as UI and internal as multiple APIs which take the input and respond for movement of the car?

So the question here is:

• Testing of functionality just for the internals of a car is enough?
• By doing so, there is no need to test for the exterior (UI) of the car?
• If yes, why would skilled people work in designing and testing the UI that is exterior of a car, constructing various scenarios?
• Testing for the security of a car internal is sufficient? Should the UI of also of a car be subjected to security tests? Ask the same for performance, functionality, and UX.

I have heard there OTA updates for the software installed in a car. This software can control the internal and external of a car is what I see.

I learn the API and UI are two individual entities of a system. Testing of one entity will not overrule the need for testing of other entities. Instead, it can complement and help testing better. If thinking testing of API is enough and no need to test UI, then asking what tests look repetitive is logical.

Why would people invest so much money, time, and resources in building the interactive UI? Why the world faces the problem with fragmentation in each segment? Having the engineering focus on the API segment alone is enough?

Why we usually have two teams working on APIs and front-end?  Why each of these teams are specialized and skilled in working at their areas and not in both?  If testing for API is sufficient, why there is a dedicated team for programming the front-end and its testing?  Answering these questions can arrive one at having a clarity.

Should the client be seen as UI?

What is UI in this context? The client end of the system will have code for:

• UI
• Rendering of UI based on context
• Processing the input
• Processing the output
• Processing and interacting with peripherals and its event
• And much more

To imagine the client's complexity, look at the exterior of your mobile phone device and how it interacts with internals for the input you provide. Think about the Push Notification you receive via an API and how it appears as UI in the notification tray. How acting upon the push notification in UI will respond? It looks simple case but not so when it comes to the handling of UI.

Seeing the client as UI and an endpoint to input the data and display is reasonable. But confining only to that will blind one. What was the need for the Developer Tool in a browser, if the API testing was sufficient? Every piece i.e. each entity in the software system is complex. Testing of one will not override the need for the other entity.

UI fails as hard as the API

I have been working on the server end and as well client end. I have experienced API functioning well but the client fails though it abides by contract. Should the client be seen as just UI?

The attributes of API will connect and communicate with the attributes of UI. If the API is functional and it does what is expected does not mean the client will be functional. The API is secure does not mean the client is secure. The API (service) is doing better in performance does not mean the client is also performing well. The UX can be annoying by the API while the UX of the client is well received. The context in which API is tested in isolation is different from the testing of a client.

To share one experience of me, the user entered the insurance details as in the insurance card. The API internally did handle the details differently in the presentation. As a result, the next time when API responded with insurance detail, the client crashed. It was the same data but presented in a different way which the client did not accept or understand. But when tested the API in isolation, API accepted the data in both formats. Then the business wanted the detail as expected by the client and not how the API presented it. Could the testing of API alone help me to learn the problem at the client end here? The API was vulnerable but not the client in one case. I used the API as an entry point for the attack. Eventually, the impact was reflected in UI with the response.

I have more such examples from my experiences, to say API and UI are two different entities. Testing for one in any quality criteria will not override the need for testing of the other. But it helps me to plan and design my tests well knowing them individually and how both interact.

How to respond for -- API Testing or UI Testing?

• Both are distinct, important, and needed
• Testing of each provide unique information and value
• Ask details for what to test and what is expected out of testing
• Rephrase the question asked to better one which is explicit with clarity
• Myths:
• Testing for API is important than testing for client
• Testing for API is sufficient and no need to test for client
• Testing for API will cover data which client do not cover
• As the automation pyramid talks of Unit, Integration (API) and UI layers, it is confused the client is UI layer.  UI is one part of the client and not the whole.
• Testing for API finds unique information which may or may not be found by testing client
• Identify what to test in these two distinct layers
• Both of these layers have their own state and behaviorial differences
• So trying to mock with stubs on other layer be it from API or from client, may always have something critical unlearned
• When testing in this way, usually the focus is one element of a software and so it is called Unit Testing
• Testing for the APIs and testing for the client have distinct weight and importance

    

2020: It Is Costliest If Not Shipped Quickly - Fixing few bugs later is okay!

When you read this post, I assume you are working in a team or organization which has imbibed Agile and its various approaches into practice.  I see it is more mindset and culture, than just saying it as a practice.  I have worked in team and projects which had Waterfall approach. I'm working with teams and projects which claims to be Agile in approach.

Here is my tweet and the question of @J_teapot.  His tweet is interesting and striking to me.  Hence, I write this post.  Thanks, @J_teapot for asking.  I enjoyed that question of yours.

Days of Waterfall

It was in early 2000s (to be precise 2005-2007), I worked in project which had customized waterfall approach.  The build to testing were delivered in iteration and not at the end.  In the course of project, it got adapted to Agile practices and as I remember it was around 2007-2008.  Yet the shipment of product being developed happened once in 3 or 4 months.  This shipment time remained as is Waterfall and in Agile.  If asked, why that slow in shipment when Agile, it was business decision.

In my initial days of career, I read this in text books of Software Engineering and Software Testing -- "Bugs are more costly to fix when found later."  The same was said by managers to we programmers and testers.  Those were the days where no much blogs and posts in web on Software Testing, Software Development and Practices.  Google, had just come in few years back, then.  It was the period where Software was getting into every spaces of business and daily life.  The pace of delivering software to various business space picked up its pace.  The technology platforms emerged and so the practices.

Days of Startup Era

In the era of technology start-up that is around 2012 and onwards, the shipment pace for "working" piece of software took cut from months to a month and to a week.  Then the era of programmers led technology start-up came in.  The idea of MVP (minimum viable product) and pitching to investors for series of funds, all came into the news.  If you notice, software was developed earlier and, in this date, as well.  The difference is the time and how it shipped, was the business call in major than being a tech call.  I see this is appropriate to remain in business, competition and for being relevant to needs of customer and market.  That change has to come in quick pace.

In 2020, this is no different.  In fact, the shipment pace has got much quicker and it goes in a sprint of 14 days (including non-working days).  You can refer to the train model of backlogs in a sprint for more details.  If did not get deployed and rolled out for install, business will be impacted as earlier.  Who answers for the investors?

The approach of software development has evolved and so the testing also has to adapt.  If had a "working" piece of deployment and serving the customers with value claimed, the team will be relieved; else, the trouble starts for team.  I have been here and I know it.  The MVP which claims to deliver the mentioned value, has to be delivered.  Whatever one builds or adds to MVP, should continue to deliver value claimed.  If there are bugs in shipment that can be bearable, then still fine unless it blocks the value claimed.

I have been in state, where the value claimed by business and MVP, not delivered after shipment and deployment.  The are several reasons as multiple works to ship one working software system.  Fixing that in few minutes and deploying it is expected besides the cost incurred out of it.  It so happens, the deployment and roll out goes while the testing is in progress or not yet started for that version of software.  In a way, it is a strategy and also a business call.

Ship it first - Mindset

In 2020, it is costliest if not shipped quickly; fixing few bugs later is okay.  That's a business call and a strategy.  We have to be there on time with service catered.  In my opinion, unless the value claimed to offer by MVP and product as a whole, is being delivered, all is in control despite the bugs in production.  That said, I'm not ignorant of bugs in production.  I will continue my work in finding them.  I have evolved to adapt my testing and skills to need of business, market and software system we build.

If asked what is right that is shipment once in months or once in a sprint (2 weeks, usually), both are right.  I hear few teams still ships once in 6 months today as well.  I work with teams which ships in a week and in two weeks.  Not to mention, I have worked where the requirement comes from business and shipment happens in few hours.  It is all about, where I'm here.  That as well defines the time available for testing.  I'm not sure what today's Software Engineering and Software Testing books say about fixing bug later.  Fixing later is not a habit but a situation needs today when it comes to business.  I wish, when a testers being technical and understands the business side, it adds value much more to the testing, team, business, organization, customers and to the product.

To summarize, it is business and market demand that has kept the shipment in a sprint.  Prioritizing the blockers (and bugs -- know & unknown) for shipment is a skill which a tech (programmers & testers) team has to groom in my opinion.  I see this is adapted in service industry and as well in the tech product organizations.

Web: Debugging the Errors using Browser's Utilities

My curiosity raises when I come across problems that has no first-hand clues.  I will get in and start debugging it, looking around what is available for first.  Here is one such case, which was mentioned in Facebook Group of The Test Tribe community.  It had a screenshot of a browser (not sure which browser) with URL and a error message for a untitled tab -- "This webpage was reloaded because a problem occurred."

I can't guess what went around there!  But I see, there should be a reload of page again.  The reason for it can be anything.  But I will take two into considerations at least to start:

1. Browser crash, and 
2. DOM failed and paint did not happen (for 'n' reasons, which is not know at time of witnessing it).

My questions to start looking for insights:

1. How will I know what actions did I make with browser?
2. How will I know what was loaded and not loaded?
3. How will I know if there was any errors in loading web page?
4. How will I know if there was a crash in browser? (Assuming this is a rare case, but it cannot be denied. I have witnessed browser crash when I tested for cross browser compatibility of web application)

Typically I prefer the browser in this order when I want to assess the event actions and performance -- Chrome, Firefox, and any other browser.  Note that, how the event actions and performance are handled differs from browser to browser.

Opening below Chrome URLs in a tab of Chrome, and in another tab using the web applications records and collects information that can be useful to debug.  That way I don't have to make note of my actions in parallel, while I test in this context:

• chrome://user-actions  (This records what actions I'm doing with my open browser window instance)
• chrome://history   (This shows the places I have visited)
• chrome://net-export  (I use this when I actually need it to debug much deeper, else I will not enable it.  This can have influence on the performance of a browser when enabled.)
• chrome://crashes   (lists the crashes of browser)

Make note of what you are collecting in file when used net-export utility in Chrome.  The credentials and private can get recorded as well in a file.  If this is to be shared with someone, know the risk of doing it.  Likewise, in Firefox, referring the "about protocol" available is useful. For example, about://crashes

To know if the resources was requested and was it loaded or not, below commands in console of Chrome and Firefox, helps:

• performance.getEntriesByType("resource");
• performance.getEntriesByType("navigation");

I have not found IE giving this detailed information. Or, I'm not aware of it.  If you are aware of it, please share.  I have not debugged much in Safari in recent times.  Usually, the technical heuristic is if "looks to work" in Chrome, "more likely it looks to work" in Safari.  Need to look at the CSS particulars, in specific with Safari.

These are few things which I will have to keep pre-setup, before I test web applications in browser.  Possibility of getting the required information is high if had this setup -- to debug and report bug with tech details.

In IE, I will debug from the point where the error occurs by following the stack trace details.  When done together with a programmer, it helps very much mutually.  Apart from above said, searching in web, I found several reasons stated for this error from cache to invalid time in system where browser is being used.  I will cross check quickly for them as well, while I have this pre-setup done and collecting required information.