Understanding Web Services, APIs and Architectural Pattern

Since I started my career in testing, I hear and see these words frequently -- 'Web Services', 'APIs', 'Service Oriented Architecture', and 'SaaS'. In recent seven years, I heard word 'Microservices' quite frequently. 

In overall, when I talk to practicing testers, we hear this question during discussion in knowing the differences and understanding of same.  I too had this confusion and questions.  I worked upon it to know them and trying my best to understand what they mean. Today I test them and continue to update myself while I learn what they mean in its existence and purpose.  

In this post, I share what I understand for them by saying the differences I have figured out.  So that, it helps other testers or SDETs.

Web Services

Before getting into word 'Web Services', I want to understand what the word 'web' and 'services' mean. The word 'web' - interconnected systems over network. Then what do word 'services' mean in this context? I learn - services is the endpoint of a connection; it has computer systems to support the connections that it offers.  Then Web Services is - the technologies that allow for making the connections over web and its protocols. Thereby we connect the services together using Web Services.

The Web Services are offered by making use of:

• SOAP (Simple Object Access Protocol);
• REST (REpresentational State Transfer it is a representation style of architecture, and not a protocol );
• and, other web technologies which includes protocol and architecture pattern

Service Oriented Architecture (SOA)

SOA is collection of services, where services communicate with each other.  It can exchange data from simple to complex form between one or more services to accomplish a purpose of activity (i.e. request).  Now we see that service - i.e. a function to be well thought, defined,  and not to depend on state and context of other services.  The service will have consumer (consumer of service by requesting for it) and provider (provider of service as response to request).  Note that service provider also can be a consumer based on contexts.

With this I understand, SOA is a software architecture pattern. The service provider provides the service to consumer over communication protocol.

Note: Few protocols which are commonly used - HTTP and SOAP.  I remember the days where I tested CORBA based services.

Application Program Interface (API)

Thinking in this perspective, I believe, it helps understanding it better -- information hiding. The 'hiding' here means, independent modules which can communicate with each other and exchange the only information which is required. Thereby accomplishing the modularity with independent modules.

This enables for the creation of interface which can be used by other software systems (i.e. service consumers) to access the data and update if required. 

The APIs usually will be implemented using Web Services with help of SOAP,  HTTP, REST and other protocols and architectural representational styles.  When said REST APIs, it means the APIs adhering to the standards of REST.

Microservices

It is an architectural style which structures the application as a collection of services. That means, an application multiple services which are -- independently deployable; not coupled to other services very tightly; which can be tested easily in isolation to other services; easily maintainable by a programmer; and it represents the business components i.e. business's services.

For example, the search in https://amazon.in, is one service; and, the order fulfilment is another service. Either of these service is being down does not affect other service in its functional operation. This is one benefit of microservices architectural style. 

Pictorial Representation

In the above pic,

• how two end points (service consumer & service provider) communicates is with the help of Web Services. Web Services is a concept how two end communicates in a view.
• Underlying system and data it processes, is not visible to service consumer. The service consumer just sees an interface i.e. API through which the specific service's request is made and it gets the specific response. The API does not expose other functional behaviour of underlying system and data it processes.
• The way these services are structured and organized forms architectural pattern of the system. It can be any architectural pattern and few commonly heard are -- SOA, Microservices.
• How consumer talks to interface (API) changes quite often and not the interface. Likewise, how the service has to be provided by service provider changes than the API i.e. interface.

Relating a daily life example

Air ticket enquiry and booking system:

• Airline and consumer are two end points. They can communicate to each other via phone call, email, chat, or in person.  There is a protocol which defines how this communication to happen between the two end points.  A service is provided to a consumer.  In tech, relate this to Web Services.
• Different protocols for different communication types i.e. I can't communicate as I do in email when I talk to airline staff in airport.  Relate this to SOAP or REST, to start.
• To enquiry about my customer membership with airline, I need to talk right person. A front desk staff might not be able to provide this service.  Relate this to API i.e. specific service request and an interface to it.  I don't see how they the info to me; all I see is the details of my membership and share it with me.
• If the check-in system of the airline is not functional, still the ticketing system of that airline can be used to buy a ticket. This is an example of microservices assuming 'check-in' and the 'ticketing system' are two microservices which are not dependent on each other, though they interact to complete the business need.
• Having distinguished services and staff assisting for it in airport, so that consumer can go and consume what is needed, will sum up to Service Oriented Architecture style.

The Test Data Stopped Me From Committing a Blunder

Recently, I took up testing of a feature and it was one of the critical release from point of business. We had to create new systems technically and associate with existing system for a flow to complete the business need.  This involved testing from at least 5 different end points other than different client end points.  But how to test it with the time I had? 

To test this effectively, I could have made use of existing data from stage database and mapped it or dumped it to create a copy for new system.  Then, I thought, let me create a fresh set of data.  The challenge for me in creating test data was:

1. I did not understand existing test data as it was not that straight forward and simple; most data were junk i.e. entered randomly; and with automation aid to have some data, probably
2. It has to sample for 100000+ entries in production and its mappings to different fulfilment
3. It has to help in testing the newly built system in unit wise and integrated system in each node wise
4. It has to help in figuring out the problems in each microservices and different client nodes
5. I had time of 2 days to prepare the test data
6. I had to understand the logic of how various associated microservices evaluates the same data for fulfilling the request
7. I had to understand how the data has to be constructed such that when it is fed to cron job or scheduler, it will just pick it and insert them in the database and then associate the mappings
8. How to make sure this mapping is done right? How shall I test this mapping?
9. I will be getting almost 1 day to test this new feature on one client node after data creation.  How do I make sure the client end point is dealing it right along with other client nodes and microservices end points?
10. The products get shipped at the day end of that -- one day testing of a client node
11. What is the space I have, to correct my error If I did in my testing now?
12. I was said not to test anything and just see if that functional flow is good enough to make one as usual fulfilment; should I take this and just do this and nothing else?
13. I said to team that no junk data entry and term this as test data; I had stopped team from entering data for two days
14. Data being created from automation in systems was posing challenge each minute in this time and I had to collaborate with different teams on same while I had left with little time to complete tasks

I leant, my testing can be strong here if at all, only if I understand data and how it gets fulfilled.  I decided to sit and create the test data.  While I created test data, I started referring to production environment and created the data to simulate them.

I prepared 42 test data and I knew exactly what it is and I had sampled it enough to simulate 100000+ data.  I had taken close to 3 days for designing these data and associating it without help of any cron job or scheduler.  I figured out how to evaluate these data in each nodes and microservice end points by this time.

This is the advantage of the test data creation. I spent most of the testing time here i.e. in preparing the test data. It helped me very much!

With this, I had set of tests which are "must" and any problem in these, it will be blocker. In other set of tests, if there are any problems, then it can be called out by stakeholders if that has to be fixed or not for releasing the product.  I was left with 3 hours before they take call of shipping the product, that day.  I had to cover my tests by this time.

It was about 2 hours left for shipping decision, I noticed a behaviour which seemed to be release blocker. On informing it, I started looking at different client nodes to see if that exist there as well.  I could identify this blocker, because, I had designed the test data. If I had not designed the test data, it would have not been possible for me to identify it that day. One of the disaster probably in the product's history if that had gone out.  The functionality seem to work and fulfilment runs. But the feature is not doing what it has to do.  See how tricky it is - it functions but it does not do anything what it has to do eventually. The disguise!

If I had just carried out the functional flow per communication I received, I would have done a blunder as practicing tester. The business does not do this purposefully; they assume it will be handled and there is no need to test this.  If I have to make the same decision as a tester i.e. as business, I should be convinced enough to say, there is no need to test this at this point of time given the context. Good that, I took it up by making time to test each end points. 

The functionality flow was good but the outcome was not what business wants and it is not obvious anywhere in the UI.  Debugging this behaviour, I learned this is problem. I figured out the root cause of it and shared it to the programmers.  I was given a new deployment to test for the same, later.  The shipment of product did go on same day.

Why did I share this here?

1. Test data is important part of a test
2. Time what we spend on creating test is part of the time which we take for testing
3. Sampling and creating the test data is a skill to be improvised each day by tester
4. When said don't test this, as a tester have enough confidence and rational analysis before not testing it; else, as a tester we will be supporting the growth of product's risk
5. Creation of test data helps to know different end points of a technical system; if did not help, then we are not preparing the right test data
6. Blockers, bugs and ambiguity always exist; the tests and test data should help in making it simple to understand and know the root cause
7. If I had not worked on this test data, I would have not identified the blocker and it would not have been possible for me to test each end nodes of client and microservices

Now if this is automated on this feature, the automation can assert for same on different end nodes because we know what is the data and what is expected out of it. There is no need of me now to do this as extensively as I did this time.  This is one of the benefit of automation which lets me to focus on other stuffs in testing.

Note:

• To a practicing tester, it is good to report the problem clearly. There is no need to solve it because we will not know exactly what it is and why it is as we will not have source code with us. But as practicing gets stronger in a project or with a product, few things can become clear technically and this will lead a tester to tell what is the root cause upon technical investigation. This takes practice, yet we can go wrong. So never exaggerate the bug report saying technical investigation and root cause. Rather, we can provide, this is what we learn from our test investigation.
• Never exaggerate the test data and at same never under estimate the test data. Anything associated with test data is worth investing time to look at it and know it better.