Do Adding Cores Improves The System's Performance?

Hardware and Programming Language

If I have no understanding of -- sysem's architecture, how the system is designed and implemented technically knowing its business purpose, then I cannot test for performance rationally and technically, 

In this context, the awareness and understanding of below is necessary and important.

1. The infrastructure where the system and its services are deployed and consumed
2. The hardware on which the system is deployed
• The understanding of the hardware along with its limitation
3. The hardware on which the service of the deployed system is consumed
• The limitation of consumer's hardware
4. Understanding the CPU and its Cores
• How we are programming the threads to exeucte on these Cores?
5. The programming language used to implement the system; this play a vital role
• Does it allow the threads to run on two and more different cores at a time?
• Or, does it confine the threads to run on one Core?
6. The way in which we have programmed the system for its instruction execution at a thread (subroutine) level
• How the threads are implemented and how it exeuctes on a CPU and its Cores?

I should be aware of above said information when building high performant system and testing for the performance of a system.

The value and benefits of these information are not just limited to performance testing alone. It also helps in testing for seucrity and functionality.  The awareness of these information will give you an edge in the testing for performance.

CPU Cores and Software System's Performance

I hear this often and especially during the businness seasons time:

"We will add more CPU with more cores. This will improve the exeuction time and improves the performance.  The business will not be impacted."

Does this make sense technically?  What's your thought on above statement as a Test Engineer testing the system for different quality criteria?

If I do not have awareness on information that I said above, I will not be in a position to test and advocate better for performance.

Do Cores Added Reduce the Exeuction Time?

By just adding more cores to a CPU, it does not always speed up a program's exeuction time.   

I learn, if a program which is designed to run on multiple cores have threads, that must run on one core, then this will be a limitation for the maximum speed [by reducing execution time] which we can achieve by adding more cores.

Also, the programming language used will have a role.  If the programming language uses Global Interpreter Lock (GIL), then it makes sure that a process created out of it can run only one instruction at a time, despite of the cores it is currently using.  That is, though a process created has access to multiple cores in a given time, the insturctions will be running on just one core.  Which means, the threads cannot run on multiple cores.  The instruction will be running on just one core and just one thread at any point in time.  Eventually, this leads to higher exeuction time for a process of program.

Should I say high exeuction time is a high performance and high performant system?  That is contextual!  What do you say?  What should a consumer of the service (business) say about this performance?

To summarize, adding multiple cores to a CPU, does not necessarily speed up the exeuction time for a program.  It is dependent on how we have designed and written the program and the programming language used.

To know more about GIL

• Global Interpreter Lock -- https://en.wikipedia.org/wiki/Global_interpreter_lock
• https://langdev.stackexchange.com/questions/1873/what-is-a-global-interpreter-lock-and-why-would-an-interpreter-have-it

Logarithm and the Expression of an Algorithm

I got to know about Logarithm in my High School.  But, I never knew it would be part of an engineer's life.  As a Software Test Engineer, I encounter the discussion that involves Logarithm often when testing an algorithm that is designed and implemented to solve a problem.

What is Logarithm?

I found this definition in Math is Fun. And, I see this is simple and straight.

How many of one number multiply together to make another number?

For example, how many of 4s make 64? 4 x 4 x 4 = 64.  

That is, when multiplied 3 of the 4s, I get 64.  Therefore, the logarithm is 3.

It is represented as below and said as "the logarithm of 64 with base 4 is 3".

• log₄₍64) = 3

The same can be written or expressed as exponent in Math.  That is, 4³ = 64.  The exponent and Logarithms are related.  The logarithm tells us what is the exponent.  

The logarithm answers the question -- What exponent do we need for the chosen base to get the given number?

John Napier introduced Logarithms in 1614 as means to simplify the calculations.

What Logarithm?

In the Computer Science, we use the Common Logarithm.  In my practice so far, when talking about the algorithms, I have been using Common Logarithm which is denoted using log.

The other type of Logarithm is, the Natural Logarithm. This is denoted using ln.

What is the Base?

One of the questions which is asked in discussion of an algorithm's analysis is what is the base?  This leads to question, What is the base that we should consider in Computer Science?

In my so far experience, I see, it depends on the context of an algorithm that is under evaluation and if it is of logarithmic nature in the time complexity and growth rate.

In the context of Computer Science, I see, the base do not matter much when equating the right hand side to the left hand side calculation.  But, one can pick the base that suits to context when needs to express the relationship of an algorithm under evaluation. The base need not be always 2 here.

Note: I understand below from the peer discussions for the logarithmic base in Computer Science:

The asymptotic notation focuses on growth rate and ignores the constant factors. Any two logarithms differ by a constant factor, and the base makes no difference.  Hence, I do not see base specified for a logarithm when using asymptotic notation.  That said, when I have a logarithm with a constant base, it is okay to not specify base.

When the base of logarithm depends on parameter (an input or any external configuration) to the algorithm and which is not constant, it is a better practice to mention the base.

In my testing experience for an algorithm's functionality, performance, complexity and growth rate, I see, the engineers keep the input parameters as configurable.  That is, it is kept as a constant which can be changed based on the need basis for the context.  So the base is not mentioned in the logarithmic expression for an algorithm most times.

If you see, my understanding is not appropriate in context of logarithmic asymptotic notation, do share your thoughts as comments to this blog post.  I will be happy to introspect and learn.

References:

• https://www.mathsisfun.com/algebra/logarithms.html

Performance Test Report: Between the Effective and Ineffective Reports

 

In this post, I'm picking the thirteenth question from season two of 100 Days of Skilled Testing.

What is an effective way of reporting performance test results and mention some tools you have used in test execution, analysis, and reporting?

I see two questions.  I see it is not a wise attempt to learn these two questions combined as one.  In my opinion, the second question added, it dilutes and make the whole question vague.

What Should a Report Do?

The report should be contextual, compelling, influencing, and targeted to the intended audience to act upon on making a decision.  The software testing report is not an exemption from it.

The performance testing report should know

• Who are its audiences?
• How they read, relate and understand the information?

If this is ignored, the report will not serve the purpose of commissioned testing.  The effectiveness of a report cannot be determined solely on how the stakeholders responds to it.  

On understanding the risks and problems in the system's current capability, mentioned in the report, the stakeholder might not respond with an action to tune the performance aspects.  This could be for multiple factors including that of business.

Note that, a skilled and problem solving engineer understands the business and how it drives.  Just being technically skilled will not help an engineer to grow in a longer run in her or his career.  The system's performance tuning decisions most times will be driven by business.

Did the report persuade the stakeholders with an awareness, mutual understanding and agreement?  The report should drive this conversation.  If not, we have a problem.

On reading the performance testing report, do the stakeholders get an informed awareness on what happened during testing in the present capability of a system's performance criteria?  Do the stakeholders understand and mutually acknowledge how it benefits and costs the business?

This is the foremost value serving expected from a testing report.  If not, I look at how the data and story is presented in the report.

The bottom line is, did we mutually acknowledge, agree and understand on current capability and consequences?  If not, the basic purpose of the report is not met.

Performance Testing Report

The software testing is a high technical activity.   You agree or not to this, but, this is the reality.

Testing for performance is technical investigation activity. It includes the orchestrated study in correlation of 

• hardware, operating system, network, tech stacks & software used in SDLC, architecture, designs, certain decisions, people, business and you - the test engineer.

The fundamental in-depth awareness and knowledge of these areas is essential and a necessity to analyze the performance's aspect.  The performance testing report will show this trait of you as a test engineer.

We have stakeholders who work in technical area and in non-tech area.  How to compile the effective performance testing report?

There is no one way or defined way of writing an effective performance testing report.  Figure out what works in context of your testing to have a effective report knowing - What Should a Report Do?

Outline of Persuading Performance Test Report

It is a technical story telling in a non-technical way with data, pictures, comparison by relating, metaphors, and contemporary history.  I compose the performance testing reports in line with business targets and objectives set.  I provide a metaphors to relate and know the value and cost.

At times, I will have two reports.  I share it with respective stakeholders.

• One with non-technical summary and conclusion
• The other with technical details, analysis and data from investigation

Sometimes, I include the above two reports in one report based on the context.

In overall, this will be in minimum as part of my performance testing report to start.

1. What part of the system is being tested?
2. Why that part of the system is being tested?
3. Mentioning the vague performance requirements gathered from stakeholders.
4. Refining and precising the performance requirements to be specific, contextual and deterministic.
5. Who are the stakeholders of this report?
• What sections the respective stakeholders to refer for the analysis and outcome?
6. Problem statement of the performance testing statement
7. Brief summary of performance testing outcome. [TLDR]
1. What aspect of system's performance is evaluated and why?
2. Brief summary of performance test carried out and outcome.
8. Detailed Report with Technical Details
1. Analysis and Technical Investigation
2. Representation of data which is analyzed
3. Identification of bottlenecks, risks, problems and its symptoms
4. Summary of the test's outcome

You don't have to stick on to one format or a template.  Figure out what works well in your case so that the intent of your tests and outcome is understood by stakeholders.  Give a structure to your report!

The performance testing reports will have metrics, graphs, numbers and proposals.  The presence of metrics, graphs, numbers and the other said, does not make the report effective.  Then, what makes it effective?  When you call it effective?  When you accept it is not effective?  Only, you can figure it out to your context.  I can assist you here; pull me in.

There is no good [effective] report or not good [ineffective] report.  The report is either

• From a team with skills, experience, trained, and practicing
• From a team which is not trained, and, not practicing

Performance Testing - What to Know Before User Behavior and Traffic Pattern?

 

This blog post is in series of 100 Days of Skilled Testing.  I see, I do not have to pick every questions asked in this series.  I pick and share to which I see, I can add value.

The twelfth question from the season two of 100 Days of Skilled Testing, is:

What strategies do you use to simulate realistic user behavior and traffic patterns when conducting performance tests?

The twelfth question asked is vague and it needs to be refined for preciseness to pick it up and continue.

The Question and the Gap

I see the below are missing in the above asked question:

1. What aspect of performance is under evaluation?
2. What is the system that is being evaluated for a performance's aspect?
3. What part of the system is being evaluated for a performance's aspect?
• Queuing? Messaging? Database I/O? Memory? Space? CPU? Client Performance? Functional Module?
4. Who are the users?  What are their personas?
5. How and where the users are accessing the system?
6. What is the context of users accessing this system?
7. What is the geo location of users who are accessing this system?
8. How long these users are connected by accessing this system?
9. Are there any differences among these users in their roles and privileges in accessing this system?
10. Can the user access system through multiple interfaces?
11. Are you assuming the user is on web browser and mobile apps to access this system?
12. Is this system you are referring to, is a software system? Or any other system that is controlled environment like - access door, elevators, etc. ?
13. You are asking to simulate the user behavior and traffic pattern.  Should I assume, I and you know or agree to any volume of user?  And, all these users are here for the same purpose when accessing the system?
14. Are you considering any time or at a particular time when talking about the traffic pattern?
15. Are there any unrealistic users who is accessing your system?  You say 'realistic user'.
• Do you see that bots and non-human are also allowed as a user in your traffic?
16. Have you evaluated this earlier in your system?
• If yes, do you have the history and data for user behavior and traffic pattern?
• If you don't have, do you allow to use or have your competitor's user behavior and traffic pattern data? 
17. What is the tech stack of your system?
• What part of your tech stack, you want to evaluate with this user behavior and traffic pattern?
18. What is the architecture of your system?
19. What part of your system and its architecture is being evaluated with this user behavior and traffic pattern?
20. Are you running this exercise for the first time?  If not, where I can refer to previous exercises?
21. How the interaction and events are handled from its start to completion?
• What all are needed to complete the transaction in work flow?
• How this transaction can go invalid for lack or incorrect data, state and action?
22. What is spike, drop, saturation, expected, unexpected, and average numbers in the traffic coming in?
23. What do you understand by traffic?  Do you mean number of requests coming in?
• Do you mean the being committed I/O operations?
• Do you mean the response received at the other end?
• What is the definition of 'traffic' in this context?
24. What is that you want to study and evaluate by the User Behavior and Traffic Pattern information gathered in this context?

Using the above questions, I will get an idea to proceed.

I will build a model from information I collect using above asked questions.  This model we will used to further in testing for a performance's aspect.  The value added to the performance test depends on this model as well.  To get a better model in context, it is useful to address the gaps.  From here, I start to think further.

What do you ask and look for when building a model for User Behavior and Traffic Pattern?

Performance Testing - The Unusual Ignorance in Practice & Culture

 

I'm continuing to share my experiences and learning for100 Days of Skilled Testing series.  I want to keep it short and as a mini blog posts.  If you see, the detailed insights and conversations needed, let us get in touch.

The ninth question from season two of  100 Days of Skilled Testing is

What are some common mistakes you see people making while doing performance testing?  How do they avoid it?

Mistakes or Ignorance?

It is mistake when I do an action though I'm aware that it is not right in the context.

I do not want to label what I share in this blog post as mistake.  But, I call it as ignorance despite having or not having the awareness, and the experience.

The ignorance said here is not just tied to the SDLC.  It is also tied to the organization's practice and culture that can create problems.

To this blog post's context, I categorize the ignorance in these categories -- Practitioner and Organization.

1. Practitioner's ignorance
• Not understanding the performance, performance engineering, and performance testing
• When said performance testing, taking it as - "It is load testing"
• No awareness on what is performance and performance engineering
• Going to the tools immediately to solve the problem while not knowing what is the performance problem statement
• Be it web, API, mobile or anything,
• Going to one tool or tools and running tests
• No much thinking on how to design the tests in the performance testing being done
• Ignoring Math and Statistics, and its importance in Performance analysis
• No idea on the system's architecture, and how it works
• Why it is the way it is?
• The idea of end-to-end is extended and used in testing for performance and having hard time to understand and interpret the performance data
• How many end-to-end your tests have identified?
• Can we test for performance to all these identified and unidentified end-to-end?
• Relying on the resource/content in internet and applying or using it in one's context without understanding it
• No idea on the tech stack and how to utilize the testability offered by it in evaluating the performance
• Not using or asking for testability
• Getting hung to most spoken and discussed 2 or 3 tools on the internet
• Applying tools and calling out it as performance testing
• No attempting to understand the infrastructure and resources
• How it impacts and influences the performance evaluation and its data
• Idea on Saturation of resources
• Thinking it as a problem
• Thinking it as not a problem
• Not working to identify where will be the next bottleneck when solving a current bottleneck
• What to measure?
• How to measure?
• When to measure?
• What to look when measuring?
• Not understanding the OS, Hardware resources, Tech Stacks, Libraries, Frameworks, Programming Language, CPU & Cores, Network, Orchestration, and more
• Not knowing the tool and what it offers
• I learn the tool everyday; today, it is not the same to me compared to yesterday
• I discover something new that I was not aware of what it offered and exist
• I learn the new ways of using the tool in different approaches
• No story in the report with information/image that is self-describable to most who reads it
• And, more; but the above said resonates with most of us
2. Organization's ignorance
• At the org level, for first and to start, it is ignorance in Performance Engineering
• Ignoring the practice of performance engineering in what is built and deployed
• Thinking and advocating, increasing the hardware resources will increase and better the performance
• In fact, it will deteriorate over a period of time no matter how much the resources are scaled up and added
• Ignoring the performance evaluation and its presence in CI-CD pipeline
• The performance tests on CI-CD pipeline should not take beyond few minutes
• What is that "few minutes"?
• Not prioritizing the importance of having the requirements for Performance Engineering

Recently, I was asked a question - How to evaluate the login performance of a mobile app using a tool "x"?

In another case, I see, a controller having all HTTP requests made when using web browser. Running these requests and trying to learn the numbers using a tool.

I do not say this is wrong way of doing.  That is a start.

But, we should NOT stay here thinking this is a performance engineering and that is how to run tests for learning a performance aspect[s].

To end, the performance is not just - how [why, when, what, where] fast or slow?  If that is your definition, you are not wrong!  That is a start and good for start; but, do not stick on to it alone and call performance.   It is capability.  It is about getting what I want in the way I have been promised and I expect; this is contextual, subjective and relative.  The capability leads to an experience.  What is that experience experienced?

Sometimes, serving the requests by what you call as slow, is a performance.  What is slow, here?

The words fast and slow are subjective, contextual and relative.  It is one small part of performance engineering.

That said, let me know, what have you been ignoring and unaware in practice of Performance Engineering & Testing?

Behind the Every Test Data, There is a ?!

 

Read this blog post to have a perspective about the Test Data and Test Data Management.  The point is, if I'm not aware of a test and what does it tell me to explore, I cannot think of a Test Data.

That said, if I know what I should be evaluating as part of performance, why, when and how, this will help me to come up with a thought for identifying the tests and its test data for the same. 

The ninth question from season two of 100 Days of Skilled Testing is:

What role does data management play in performance testing, and how do you ensure the availability of suitable test data sets?

Testing and "Ensure"

We test and have tests in testing, because, there is no "sure" and "ensure" idea in software.  But, we presume on a rational basis upon, "if, these are this", in a given context when the software processes.

Now, ask yourself, how can we ensure the availability of suitable test data sets?

In my opinion, the Test Data is often misunderstood.  This is the primary problem and should be the first problem, when asked "what are the challenges in creating the test data?".

When you read the concluding lines of this blog post, you will learn why I say this.

Test Data and Immunity

In my opinion and experience in practicing the Test Engineering, I see, the Test Data should be a viral strain and it should have its variants.  When this test data is used to test [experiment, test investigate, and debug], how do the software and its ecosystem respond?

• Does the software and its ecosystem is immune to this test data?
• Does it exhibit any risks and problems?
• If yes, then, do the purpose of my testing and automation is accomplished with this test data?

This puts me back to question, what is the purpose [intent] of my test?  It drives me to derive the test data which helps me to know -- What am I supposed to learn and on priority?  With this, I get an idea for what kind of test data I should be creating knowing its pattern.

If the system is immune to Test Data and not reveling anything new in the context, I classify this pattern of test data as "Immune" to the context.

In my practice and research work in Test Engineering and Software Testing, to start, I categorize Test Data into two areas.

1. Immune
2. Not Immune

Further, I have categories, under these two, where I classify the Test Data deterministically for the context.   Get in touch if you want to learn more about this.  I'm just one ping away!

The tests should help me to evaluate for the immunity and also non-immunity; both are essential and necessity.  

The credit is to me for such classification of Test Data.  It is my research work out of my practice.

Note that, Test Data is not just the input [characters or files] entered or given to a system.  Test Data has its association to tech stacks, infrastructure, ecosystem, business workflows and people.  To craft such Test Data, one has to have the understanding of the system and its internals, and, the problem it solves by knowing how it solves.

Performance Testing and Test Data

1. What is that I'm testing as part of performance?
2. What do I want to evaluate in the name of performance?
3. What part of the system is evaluated for its performance?
• Should I evaluate this in isolation or as a wholeness of the system?
4. What domain knowledge and information I should have when testing for performance?
5. What system's architecture and internal details I should understand and be aware to test for performance?
6. Is this the first delivery?  Or, do we have this system running in the production?
• If it is first delivery,
• How will I create the test data to suit the consumers of this application?
• What are the key workflows of business that we should be evaluating for its performance?
• Do all workflows and sub-systems need the evaluation for performance, and on priority?
• How do I map the fragmentation of users and their data [with its patterns]?
• What are the infrastructure and ecosystem characteristics that should be part of the test data identified?
• Does caching have any effect if the same pattern of data is used?
• If it is a running version in production
• Can I refer to the DB to figure out the pattern for the particular workflow that I'm evaluating?
• How can I match the test data to have the production data's characteristics and attributes?
7. What is the backup strategy for the Test Data?
• How do I version control the Test Data?
• Which version of the Test Data I should be using?
8. What is the threshold I'm targeting with Test Data?
• What should be the size of the data in DB when I make the IO and RW operations?
• What should be the network capability when I make the IO and RW operations?
• What should be the hardware capability when I make the IO and RW operations?
• What should be the geographical traffic and its pattern when I make the IO and RW operations?
• More of such factors will be considered when identifying and deriving the test data.
9. What is the client error yielding Test Data that I should have for the workflow?
10. What is the server error yielding Test Data that I should have for the workflow?
11. What is the redirection yielding Test Data that I should have for the workflow?
12. What is the no-response and no-change Test Data that I should have for the workflow?

And, more.  It is simple; get in touch to discuss and know beyond the listed.

To conclude and stop here, all these questions, do not ensure or assure or make sure that I will have test data for evaluating a characteristic of performance.

• It helps me to know:
• What are the tests I should be doing?
• What kind of preparation I should be having in my practice to create the Test Data for these tests?

The, Test Data should challenge the available Testability and its limits.  If it is not doing, then, we are having a test data no doubt about it; but, it is of shallow. Shallow!?

One has to ask self, "Is this sufficient enough and effective Test Data for the system [and workflow] I'm testing?"

The, Test Data should drive the engineering team to add more layers of Testability into the system.

Waterfall or Agile: Testing for Performance - Where to Start?

 

Do you understand the Agile?  I have shared my understanding here; give it a read.

The eighth question from season two of 100 Days of Skilled Testing is:

Can you share some best practices for conducting performance tests within an Agile development environment?

Best Practices and the Agile

The irony is, the Agile says, there is no best practice.  It asks, to tailor and fit the practice to the context so the continuous delivery and value is delivered consistently upholding the Agile's principles.  

Yet, we talk about the best practices in the Agile's context, like the eighth question asked here.

What is the effective way to test in the continuous delivery?

As a test engineer, how can I start thinking and testing for performance from the inception of a feature's thought?  I see, it is not hard to do so.  As you read further in this post, you will have a perspective and awareness to do it.

Performance in Waterfall and Agile

I learn, the performance is an experience.  It does not differ because of the Waterfall or Agile.  If the performance is not a pleasing experience, it will impact stakeholders no matter it is Waterfall or Agile.

But, the question when evaluating for the performance is -- where to start, when to start, how to start, and with what to start?

As of today, I do not see differences in the mindset and skills one has to have for testing of performance in Waterfall and Agile.  Could be the approach differs in certain phases here; otherwise, I see the same in both practices.

I will rephrase the eighth question to this:

What is your practice to evaluate the performance right from the start of product development in your project?

I do not want to wait until to hear -- the development is completed and deployed; now we can start running the performance tests.

What can I do as part of performance tests from the first day of development and first commit?  This is my intent and area to look in strategizing the testing and tests.

The Culture of Engineering

At the start and end of the day, when we developers start and finishes the work,

• How the work is done and why, is defined by the engineering culture practiced by that organization.
• The Performance Engineering of the software products and solution being built will be driven the by the culture practiced.

The Test Engineering and how we test and automate will be driven by the culture of engineering practiced in the organization.

Writing the code not just for building the functionality, but, also for performance is a culture driven factor.  The organization's culture for engineering practice drives it!

Testing for Performance - Where to Start?

I'm sharing my research work that I'm doing and experimenting on performance engineering and performance tests.  I'm seeing the results and value out of it and so are the stakeholders.

Today, we are getting skilled in exploring and testing without the requirement document and SLAs in hand.  Isn't it?  Haven't you?

I use my MVPT to figure out what are the minimum performance tests for the feature.  As part of this, I will explore with help available aids to evaluate the performance.

To start, I will use these questions to figure out the performance tests:

• What is the minimum viable questioning performance tests that you have got to test this feature?
• What is the minimum viable questioning performance tests that you have got to test this workflow?

Unit Tests for Time and Space Complexity

I will work closely with programmers to gather information on below when the code for the feature is committed as part of Unit Tests.

• The execution time taken by the code of that feature - the Big O Notations for space and time complexity
• Usually the Unit Tests focuses on functional tests and clean code practice
• But, when we test team ask and push for performance data, this can come as part of Unit Tests
• An architect or a principal engineer can set an expectation on
• What should be the time and space complexity of a code for a feature?
• Each functions and blocks need to be evaluated on this
• As said earlier, this depends on a engineering practice culture of an organization
• If the culture wants it, it will be there; else just the functional code will be delivered and not the performance code
• If the time and space complexity analysis outcome is not as expected, the code written has to rethought and refactored
• The review process need to put it back
• The comment with data has to be published
• This will be useful to model the performance tests by test engineers who will be working on performance tests
• Doesn't it look a like a effective useful practice as part of Performance Engineering right in the early stage?
• This is very well applicable to projects running on Agile or Waterfall

Do you have this in your project and Unit Tests written?

The time and space complexity questions should not be confined just to the SDETs [test engineer] interview.  A test engineer has to ask for it and apply it in her or his day-to-day work.

Profiling Tests by Test Engineers

We testers do not get into product's code analysis.  We have to build skill to run the profiling on product's code and analyze the resources data.

• Test Engineers can test the feature's code with the help of IDE's profiling (runtime analysis) and collect the performance data by identifying the performance bottlenecks
• This runtime analysis can profile for
• Memory snapshots
• Thread analysis
• Monitoring resources
• CPU and allocation profiling
• And, more
• The problems and risks can be reported upon analysis
• Compare the two different solution's approach performance data

This information will tell and indicate where is the risk and problem when we deploy the code.  In my opinion, this is a useful information in modeling further performance tests.  This information is first-hand information which is very powerful before we start using any other performance testing strategies and tools to aid the tests.

Get Started with Performance Engineering and Tests

These are available in the IDE.  We think of performance testing tools and ask how to test for performance.  To be precise, we test developers (test engineers) should change our mind and shift for first.  If not, as I say, we will be the bottleneck for first to ourselves.  Did you know this way of testing for performance?  Why not you introduce this in your project and organization?

If seen, these test practices can be used right from the day we commit the feature's code. This is a place to start for the performance tests.   This will be a differentiator together with MVPT and guides the MVPT to design effective performance tests in the context.

I do not say these are best practices and there is no best practices.  But this is a useful practice when the organization and stakeholders ask for it.  Let your organization and stakeholders know how well you can test for performance right from first commit of product's code.

To stop and end here,

1. Just do not test for functionality from day one, also test for the performance from the day one.
2. Influence your organization's engineering culture and developers not just for developing functional code, but, also for the performance code

Is Performance a Perception to an Engineer and User?

 

When hearing about performance from customers and engineers on team, I see each are having a perception of it on using a product.  To one it was fast enough, to another it was as usual and for one it was too slow.  Each are expressing their perception.  But, what is the performance?

What is Performance?

I see, understanding and knowing "What is performance?" is important for everyone who is involved in building the software system and product.  In my opinion, this should be the start point.  It is beneficial, when everyone has a shared common understanding to it as a team and business.

Performance is an emotion to a user!  Technically, the performance has multiple facets to it for understanding the capability of a system and its sub-systems.

Facets of the Performance

What facet do you consider and call it out as Performance?  This is another point where most of us do not align.  For example, below are the different facets that we usually hear and read a lot:

• Heap and Memory
• Threads
• Not supporting for concurrency
• Not handled well in concurrency
• Holding up other processes and causing bottleneck cases
• Memory Leaks
• Concurrency
• Memory not reclaimed
• and, more ....
• CPU consumption
• Open connections and its I/O
• Held up in processing requests
• No enough resources to process
• and, more ....
• DB I/O
• Open connections
• Unindexed data
• SPs and Queries holding the transactions
• Incorrect configurations of DB Server and nodes
• and, more ...
• Disk I/O
• Running out of space
• RW I/O not responsive
• Network consumption
• Unmanageable transactions
• Transaction's data, size and time
• Latency
• Latency in which interfaces?
• Ineffective caching, queuing and messaging
• GUI not rendered or painted
• GUI exhibiting jank behavior
• GUI partially rendered
• GUI not in a interactive state
• GUI not responding to an action
• GUI and UI loading multiple times with no room for interaction 
• Terminal yet to return and show the prompt
• Older and deprecated libraries with latest dependencies
• Server, orchestration, and sub-systems configured incorrectly
• Hardware resources and its specifications used in a context
• Display refreshing rate and frames lost with GUI rendered
• Heat dissipation
• From the hardware
• Experienced in the environment
• Time taken for a request to reach the actual end point
• Execution time on receiving a request
• And, more...

Further we have classified it to frontend and backend; both are important and equally needed.  The webpage has got different KPIs and metrics to determine where do its performance stand.  Likewise, for backend.

Which facet of performance need to be evaluated and in which phase?  Why?  The perception will be established when testing, on how we test it, and how one uses the product.

With all these for performance, where to start and what to look at?  This is one of the question with which we are left in Waterfall and Agile.  That is where the eighth question of 100 Days in Skilled Testing comes in -- Can you share some best practices for conducting performance tests within an Agile development environment?

Have you asked these questions to yourself and team?

1. What is performance to you and to your stakeholders?
2. What should I consider in evaluating for the performance of your software system?
3. What is the practice I want to pick to evaluate the performance?
4. Where do I start and how?  

The "Bottleneck" in a Test Engineer's Eyes

 

Preference to Bottle Over Jar! Why?

Have you heard Jar Neck anytime when describing a problem or solution?

• I have heard Bottleneck often and consistently; but, not Jar Neck .  Why? 
• Be it in Software Engineering or day-to-day life problem solving description,
• The Bottleneck is referenced and not a Jar Neck.

Looks like people want Bottle but not the Bottleneck speed and benefits.  Bottle without its neck is a jar?!

Bottleneck exist for better controllability.

• In a bottle, the bottleneck is a solution!  It is not a problem!
• It is to mitigate any risk and problem that arises from the flow of content in the bottle.

Yet we describe, learn and communicate the neck of a bottle as a relativity and analogy to a problem.  

Are you aware of Gateway in the software system?

• The Gateway can be seen as a neck of a bottle which controls the incoming requests and outgoing response.
• Gateway is a necessity.
• We need Gateway to be adaptable in size of its neck based on traffic volume it is handling.  Here, the gateway's neck size should adapt and scale contextually.
• When describing a problem, we are talking about how this bottleneck size which is not adaptable for the context.
• That adaptability has to be built in engineering to scale in any dimensions and magnitude.
• When this is not done, we equate the software system's problem to a bottleneck as a analogy, which is incorrect!  The bottle has got its size and its neck size fixed for a purpose and as a solution.
• The context of a bottle and today's any systems are different.
• It is good to draw similarities from General Systems Thinking and observations.
• But the solution cannot be generic to all systems; it has to be contextual.  The software system has to have its contextual solution.

So, next time when someone in your team or network talk about bottleneck, do share them bottleneck is for better controllability.  Having a contextually resizable and adaptable bottleneck is the need for Software Engineering; not the elimination of bottleneck.

In fact, a software system should have and will have a bottleneck in a point.  And, this bottleneck will be adaptable to the context for having what it should let through and process.

Is the runway of an airport a bottleneck when it is compared to a sky?  Is that a solution or a problem?  Likewise, the ship will have a defined route path and it does not sail without a route path.  Is this a bottleneck to ship and its business?  A elevator can accommodate the defined number of people or kilograms allowed, and not beyond that to move.  Is that a bottleneck?  The esophagus in human body has a size which medical science observes as normal and acceptable; any deviation from that size measurement, the medical science test investigates it as a risk and problem. Why?  Is the circumference size and length of esophagus a bottleneck to human anatomy and physiology system?

The engineering solution will and should have a bottleneck at a point.  Having a adaptable bottleneck to the context is one what tries to accomplish in a software system's scalability and operability.

Please, do not equate solving a "bottleneck" situation with Agile practice.  Does it look like a joke?  I will not be surprised if someone says bottleneck problem is solved if practiced Agile.

The Internal Metrics of High Performance Mobile App

 

Do you have a question -- What are KPIs and Metrics and the differences between them?  Then read this blog post.  It will help in knowing the differences and how each help the other mutually and remain exclusive.

The sixth question from season two of 100 Days of Skilled Testing is:

How do you determine the essential performance metrics and Key Performance Indicators (KPIs) when assessing the performance of a mobile native app?

Why I Focus on Metrics Here?

Here, in this blog post, I will focus on Metrics and not KPIs.  If you have read the above referenced blog post on KPIs and Metrics, you will learn,

• The KPIs are defined objectives by the business.
• Knowing this is important.
• But, to accomplish the KPIs objective, one has to extract the contextually suitable metric from the system and have to evaluate it.

As a test engineer, I evaluate from technical perspective with an orientation of business thoughts.

• So, the identifying and defining the metrics makes more sense to my role.
• Hence, I pick the Metrics.

A KPI example for a mobile app,

• The set percentage [what percentage?] of five start rating in store with positive emotions in a review.

What may be the KPIs, I will have to correlate it with [a] metrics so that we work towards accomplishing the KPIs set.

What is Performance to Mobile Native App?

The Native App

A native mobile app is an app developed for a particular mobile device or platform like Android and iOS.  The native apps developed for one platform cannot be run on another platform.  That is, Android app cannot run on iPhone, and vice versa.

The Android apps are written in Kotlin, today.  The iOS apps are written in Swift.  The native apps can be developed in a way that it can run both in offline and online mode based on its business objectives.

An example of the native app which we can easily understand is,

• WhatsApp for Android and iOS devices

Performance and Native App

For first, my mobile device is not my customers device.  The Android device fragmentation makes it much more challenge with computing power offered for devices by OEMs.

Though, iOS have its fragmentation on OS version and device's computing power, it is not huge when compared with fragmentation of Android devices.  This is a challenge for Android mobile app in all aspects and especially in performance.

The performance will indicate different advantage, risks and problems to a native app on a platform and its devices.  The performance can be classified into different areas for native mobile apps.

It can range from, being deep technical areas to the experience a user expresses in using the app in a given context.  Everything is performance here!

Hence it is not easy when talking about performance in the mobile's app space.  While it is so ambiguous and hard subject, how one can pull the metrics for the performance here?

From a technical perspective, the word "performance" is not specific; it is vague.  If one says, the mobile app is performing well, what does it mean?

• Is it fast?
• Is it consuming low battery power?
• Is it consuming less memory?
• Is it not consuming much network?
• Is it smooth to interact, responsive, and no jank experience?
• Is it intuitive and secure?
• Is it number of crashes?
• Is it having a consistent update and bug fixes?

What you say?

All these leads to a question - What is a high performance app?  You ask this question to yourself and to your tech team, business team and consumers.  Figure out what is a high performance app to them.  This helps!

The Common Metrics - Android and iOS

Below are some of the common metrics for which a Test Engineer extracts data and evaluate it via testing and automation.

1. App Install Time
2. App Launch Time
1. Cold Start
2. Warm Start
3. Hot Start
3. Private Memory Size of App on available Heap
4. Number of Views
5. Garbage Collection - Frequency
6. Data Residual Size and Sharing
7. Network Payload Size
8. Energy Consumption in Workflows
9. Wakelocks and its Impact
10. Frames Skipped
11. Open Threads and Processes
12. Storage & Data Size
13. App Size

And, more.  Each of these are own deep area for analysis and tuning the performance.

I have been practicing this for last 11 years in my testing.  It is one of my research areas in Software Testing & Engineering

These days, instrumentation also offers different metrics for the mobile apps which can correlate with the KPIs set for the native mobile apps.

To conclude, start understanding the internals of Android and iOS.  It opens up you to the performance and practice of high performance apps.

I'm happy to share if you are interested in practicing these subjects and testing.  Let us connect and converse!

What are KPIs and Metrics?

 

I use to have this question - Are KPIs and Metrics the same?  Especially when I started to learn and practice the Performance Testing, this question bounced back to me often.

Do you have this question?

The Use of KPIs and Metrics

When business and stakeholders talk about it so much, there should be a value out of it.  What is that value?  Why it is important to identify and capture the KPIs and Metrics?

The KPIs and Metrics are derived from data we collect.  These data are processed to extract and normalize, so that, it is in a state as expected by the consumer for making a decision.

The stakeholders will make decisions and take actions referring to KPIs and Metrics. For example,

• The number of Daily Active User (DAU) is a KPI and also a metric.
• But, a metric cannot be a KPI
• How many of this DAU, closed the transaction within five seconds using a wallet?
• It is a metric; not a KPI.

Another example,

• KPI
• How many users installed the latest version of the mobile app and have signed in?
• If there no active users on latest version that indicates a kind of risk and problems to the business.
• Reopened Tickets in Customer Care
• This indicates there is something going wrong
• Metric
• What is the average time taken to see a streaming screen for users in 4G data network?
• If this is not captured, there is no data for business to establish a relationship with KPI set.
• Average Reopened Tickets in a customer service
• The distribution and time towards lower number
• The distribution and time towards higher number

KPIs and Metrics are not the same while both have quantitative measurement. Both are different.  Identifying and knowing the difference between them in your context is important.

They go hand-in-hand when setting a direction and action.  So that, the business and stakeholders realign to the goals and objectives defined.

KPIs vs Metrics

To conclude this post, investigate your metrics and question why the chosen KPIs.  It will help you to design your Test Models and identify the tests in given context.

Software Engineering - The Unquestioned Understanding of Client in Testing

Client - The Unquestioned Understanding

When asked or said "client" or "client-side", most of us assume or take it as:

• Web page displayed in the browser
• Mobile apps - Android and iOS apps
• Desktop applications

I see this is one of the unquestioned understanding and assumption in the Software Engineering.  While it is not wrong, the client does not always mean -- a web page displayed on browser, mobile apps, desktop applications, a terminal, etc.

The client is one of the subjects which we have not attempted enough to understand in Software Testing & Engineering.

A client is one who consumes the service in form of a response, and then does what it has to do.

Client - The Contextual Entity

The entity which takes the a client place [role] is entirely based on the context.  That web page displayed on a browser is a client per some model.  So is the mobile apps.  A service looking for data from Redis is a client too.

The client can be within the backend system which requests another entity to process its request and awaits for the response.  This client is not always a mobile app, web page on a browser, or a terminal where I'm working with commands.  Do you see that?

Next time when you hear the word client, ask for the context and know who is the client that is being discussed.

Client's Awareness in Performance Tests

By now, you should be breaking your assumption and the myths around the word "client".

In testing for Performance, it is critical to be aware who is client, when and how?  Evaluating this client will not be like evaluating a web page on a browser or the mobile apps.

The fifth question from season two of 100 Days of Skilled Testing, is:

How does client-side performance testing contrast with server-side performance testing, particularly in their objectives and area of emphasis?

Hope now you will question when said client-side performance,

• What client are we talking here?
• Where do this client sit in the system's architecture.

Based on this information,

• How the tests for performance is approached and executed for a client, differs.
• What is collected and observed to evaluate the performance of a client, differs.

This blog post is not to illustrate the different clients and how evaluate it for performance.  When the question talks about a particular client and in a context, I will share the approach, and how to evaluate the same.

To end, have we explored the clientless interface?  How to test this interface?

  

Performance & Tests: Getting Started and Data Analysis

 

On running tests,

• We will have data (information) as one of the byproduct.
• Analyzing the data of the integrated sub-systems in isolation and correlation,
• It will lead us to a technical analysis on each integrated system.

In the report, we draft this analysis along with actions to be taken.

Note: When said sub-systems do not ignore or skip the client or consumer; the system does not comprise just server.

No Golden Rule

There is no one way to do a testing.  Likewise, there is no one way or the golden rule to test for performance.  It is contextual and depends on what I want to learn.

In fact, in few contexts, we can have a value adding performance test with just one request.  Just, I should be well aware of -- what is that I want to know and learn from this test.

That said, there are multiple interfaces where we can observe, analyze and learn from the performance data collected.

The fourth question from season two of 100 Days of Skilled Testing, is:

What are your favorite hacks to analyze performance testing results and find anomalies?

Well, this question do not mention explicitly if it is for server or client or database or caching or messaging or for what interface of a system.  It is a question; but, to me it looks too generic and at a point it looks vague.  Having said this, that is how the learning journey and curve starts! 

Result vs Report

What is a result?

• Is it an evaluation after a data [information] is put to scrutiny?
• Or, the result is a data that is collected and not yet interpreted?

It depends on individual or team and how it being practiced.

The result is different from a report.

Getting Started and Data Analysis

I should know how the system architecture is designed and orchestrated with its boundaries and interfaces.  This helps a lot.  What kind of architecture is this?  Is it a monolith?  If it is monolith, my approach to test for performance differs.

If I'm asked to start the analysis of data for a system that I'm not aware of,

• I will start by analyzing the below indicators on knowing the architecture and the orchestration of the sub-systems for critical business workflows

1. CPU usage
2. RAM usage
3. Data I/O
4. Network usage
5. The Heat and sound dissipated from the hardware which holds and binds
• CPU, RAM, Data I/O, Network and tech stacks installed and configured

It hints me to look further and test investigate, when I observe:

• Having a steady consumption
• What is steady consumption in this context?
• Having a low consumption
• What is low consumption in this context?
• Having a unusual consumption spike and fall of it
• I follow the pattern to study further
• What is considered as knee, spike and fall, in this context?
• Having a zero consumption
• Having a maximum consumption
• What is maximum consumption in this context?

Having a high consumption doesn't mean a problem.  Likewise, having a low consumption does not mean all is well.  I have to uncover them to learn what it means in the given context.

In each of this, there will be a pattern.  I will learn them.  I will correlate with other sub-systems and learn what they were doing in the said timeline.

Do you recollect this line -- "the architecture should provide the Testability"?

• I wrote about it in one of the blog posts of Performance Engineering.

I refer to the below by traversing with the timeline,

• The logs by asking for it
• Data recorded
• Any APMs that are in place

I correlate all these with above said indicators.

This gives me a start. It is one of easiest start that I can have to get started with analysis.

Well this is to analyze at the server end.  What about the client [consumer] end?  It is simpler and will share in the coming blog posts.

Do you want to know more on this and other strategies that can be used contextually?  Let us get connected and converse.  I'm happy to share and learn on listening to you.  It is fun and awareness!

Prioritizing Performance & Its Requirements - The Two Engineering Tasks

  

How do I gather and prioritize the performance requirements of a student from schools, colleges, universities and society? 

Note that, I said performance.  What do performance mean in schools, colleges, universities and society?

• Any time, you asked this question to self?
• If you are living with children, did this question cross your mind, no matter in what class the children are studying?

This is not the question which can be ignored.  Also, this question is not precise and to the context.  It is the question that is resonated but has no acceptable rational base for whatever context from which it arises.  The same when it comes to performance requirements of a software system.

If you observe closely, the system in which we live, it pushes towards performance for what it thinks as a performance.  Isn't it?

The third question in season two of 100 Days of Skilled Testing is:

How do you gather and prioritize performance requirements from stakeholders and project documentation?

Prioritizing Performance & Its Requirements

If you read attentively, the title of this section says - prioritizing performance and its requirements.  I did not say, prioritize performance requirements.

That said, what is performance for Netflix is not same for the  Aadhar system.  But, both have prioritized the performance and looks to be aggressive in knowing the requirements for same.  Don't you think so?

We're in the timeline of Shift Left.

• How to shift with performance to left? 
• What all in the system should focus on the performance in the Shift Left?

MVP & Performance Engineering Story

When we are going to take MVP as early as possible to market, there is a tradeoff.  What are considered in subsequent priorities which will be compromised on negotiation by engineering and business?

The context matters when prioritizing!  Be it for performance or functionality or security or any quality criteria.

I will interpret the question asked from the point view of a MVP's deployment or publishing.

• Are you asking why I have picked MVP?
• The performance is contextual and it is based out on multiple touchpoints, its boundaries and interfaces of a system.
• I cannot talk on all those in this blog post.
• Neither, I want to talk about the KPIs and metrics.  
• I want to share which you can pick, consume, and apply in your work.

Now, we have prioritized performance for a MVP.  Aren't we?  Prioritized means, it matters, it concerns us and we are okay to compromise on few for it.  Let us jump to Left with MVP in our hands to identify the requirements of business.

As a business, we will have a rough idea on how we are pitching and selling our services and to whom.  As a test engineer, you can sense what is the key transaction [business work flow] in the MVP.  You will know the touchpoints, interfaces and boundaries in the architecture that communicates and work together to keep MVP delivering the value.  Don't you?

Say the business wants the MVP to support and serve 500 requests per second.  I should know about the 500 requests here.

Is it,

• Concurrent Requests?
• Active Concurrent Requests?

This matters!  Both are not the same.  Have you asked this question?  It is a requirement we miss to capture.

Capturing Performance Questions for a MVP

It is about the awareness for first!  How much am I opening up myself to the awareness?  This brings an energy and it is contagious.

How do I bring the performance awareness in my team so that it is engineered into the system we develop?  This is a culture drive to an organization.

Now, I know, the MVP has to serve 500 concurrent active users in a second per business's expectation to meet its reach and target.  If I do not know this, I have to capture this data, for first.  How do you capture?

A Use Case to Ponder

One use case which would trigger the spark of thinking is - How should Disney+ Hotstar's services perform to live stream the India vs Pakistan Cricket World Cup 2023 on 14th Oct 2023?

• How should it capture the performance and its requirement for this day?
• How should this system scale to crores of viewers streaming the live video of the match?
• How should this system scale for the gamification - emoji, chat and other viewers engagement during the crores of viewers making requests from client interface?

Try to play the past 30 minutes of this live video? What did you see? Why? That is part of the performance engineering strategy!

This use case opens up the different topics of Architecture and Performance Engineering. Be aware of it and explore on them.  This is not what we want to talk, now.  We want to talk on a MVP and how to capture its requirements for having better performance experience.

Step Up by 5% Heuristic

On having a test environment which is close to the production context and the test data that looks realistic, I get started.

I framed this "Step Up by 5% Heuristic" after few months on starting the practice of Performance Engineering and Testing.  I failed, and I learned. I'm learning.

I know, the expectation is 500 active concurrent users per second.

I will start to evaluate the integrated systems of the MVP for the 5 percent of 500.

• What is the 5% of 500?
• I will start with 25 concurrent active users requests for the MVP.
• I will observe the emotional experience of when using the MVP during this time.
• I will monitor and record the KPIs, and other needed data.
• Does it fail to serve 5% of concurrent active users?
• If failed, I know what to do now.  It helped me.
• This helps me to draw the requirements better and rationally for the existing system's architecture.
• If it succeeds, it helped me partially in knowing what actually I wanted to know.
• I will raise the active concurrent users to 10%
• That is, increasing it by 5%
• I repeat these tests until the MVP architecture lets me know about the requirement it needs for the performance of serving 500 active concurrent users in  a second
• Read the above sentence, again
• The tests on MVP will let me learn what are its performance requirements for serving active concurrent 500 users in a given architecture, infrastructure, and tech stacks

Beyond by 37% Heuristic

I framed this "Beyond by 37% Heuristic" after I failed in framing the tests for performance.  Talking the rationale of this heuristic is not the purpose this blog post.  Let us catch up if you are curious and interested, we will discuss on this.

Do a salary hike of 30% indicate a high performance?  I don't know!  But 30% hike is something not commonly given to all, is what I see in my career so far.

That said, this 37% has worked for me in the contexts I'm testing.  Did it serve 685 (500 + 185) active concurrent users in a second?  It helps me to draw a requirement analysis of the MVP system for this volume of concurrent active users.

Now, I will step up by another 37% of concurrent active users. That is, 870 (685 + 185) active concurrent users in a second.

• If seen, I have 1.5x traffic now.  Did it serve?
• If yes, how many active concurrent users were served in a second?  
• I will correlate the KPIs of other integrated systems of a MVP.
•  With the captured data and emotions
• This will tell, what should we expect despite what is the expectation from business
• This difference will let us know "the requirements"
• How to gather information on -- what has to be optimized, changed, reorchestrated, eliminated, included, and more.
• We start technically in establishing and framing the Performance SLAs and SLOs between the tech team and business.
• Now the performance & its requirements will appear in the dots that are,
• Being connected
• To be connected
• To be disconnected
• That does not exist

To conclude, shift wherever, take the performance engineering together! Revive its requirements to be healthier!

Note: You should read these blog posts if you have not:

1. Performance Testing: Unspoken KPIs and The Missing Correlation
2. Architecture: The Common Shared Understanding -- Part 1
3. Architecture: Its Aid in Performance Engineering -- Part 2

Architecture: The Common Shared Understanding -- Part 1

 

When we are developing a software system, the requirement from a stakeholder is not 'Fast' or 'Scalable'  or  'Responsive'.  But, the stakeholder needs it and expects it.  If you see, on a larger picture, the software system development and maintenance is a job of balancing too.

When a Software Architect [Technical Architect and Test Architect], works on architecting the software system and testing for the same,

• It is about balancing the technical aspects with the business's requirements from stakeholders.  
• Do you see that?

Knowing the architecture of a software system and testing of same is one of a primary task for engineers on the project.  Because, we software engineers have to balance it well.  Balance, what? Balancing the technical aspects together with business's requirements from stakeholders.

This blog post is part of 100 Days of Skilled Testing series.  The second question posted for season 2 is,

How important is the understanding of application architecture to do performance testing better?

 

What is an Architecture?

In context of Software Development & Engineering, the word "architecture" is one of the ambiguous words among the teams in a project and an organization. 

As a test engineer,

• Did you ever had a discussion or arguments or debate with programmer and architect?
• I had such discussions and I continue to have it today as well in the projects that I work.
• This is to know and understand
• What I should be doing as an engineer for first and as a Test Architect in the role?

The outcome of this discussion showed me,

• We all did not have a common understanding of it
• We did not share,
• "What I understood for the architecture and this architecture?"

The primary goal of a Software System's Architecture is,

• We all engineers on a project have a same understanding of it, in the aspects it exists for.
• This understanding is arrived after we have put our thoughts into scrutiny and decided that we stick on to it, so that,
• We can balance well between the technical requirements and business expectation.

Are you with me, so far?

A Software System's Architecture is,

1. A common shared understanding of what we all have for,
•  What we are developing, testing, and to about maintain?
• And, Why? Who? When? Where? How?
2. Represents the boundaries and interfaces of what matters,
• That is [to be] orchestrated, designed, implemented, how it communicates, and what it will have, and not.
• It also can show how the teams are structured and how the team and organization is organized.
• For example, in the Service Oriented Architecture, the teams are built and structured with respect to the service they offer.
• It is a model that is better than other models in a given context of technical requirements and business needs
3. The context and awareness for,
• Why it is the way it is?
• The cost and value for being so.
• What to do when it has to be changed? Where to change?
• How simple and quick to change?
• What are the cost and value for being so?
• How can I monitor and observe all these consistently?
4. The Gateway of Testability - it tells what is the Testability available for,
• Letting know what is critical and priority to test
• Where, How, When, and What tests can be framed, designed, and executed? To what extent?
• Why these tests?
• If an architecture does not talk about and do not have the Testability, we have a serious problem!  This has to be fixed for first on priority.
• An architecture has to provide the Testability and Programmability scope and opportunities to develop a software system that is of value!

For today, this is my understanding for the "Architecture".

I'm a Test Architect in the role and I expect myself to be an hands-on engineer for first.  It is a necessity for an architect to be an hands-on engineer.

Note: Read the Part 2 of this blog post here.