tag:blogger.com,1999:blog-70283992289279651102024-03-17T00:22:59.493+05:30Testing GarageCurious to investigate, practice, question, explore, unlearn and learn by thinking elementally to know how I'm fooling myself.Unknownnoreply@blogger.comBlogger195125tag:blogger.com,1999:blog-7028399228927965110.post-62122973068198098242024-03-03T19:06:00.000+05:302024-03-03T19:06:29.779+05:30Performance Test Report: Between the Effective and Ineffective Reports<p style="text-align: justify;"> </p><p style="text-align: justify;">In this post, I'm picking the thirteenth question from season two of 100 Days of Skilled Testing.</p><blockquote><p style="text-align: justify;"><i>What is an effective way of reporting performance test results and mention some tools you have used in test execution, analysis, and reporting?</i></p></blockquote><p style="text-align: justify;">I see two questions. I see it is not a wise attempt to learn these two questions combined as one. In my opinion, the second question added, it dilutes and make the whole question vague.</p><p style="text-align: justify;"><br /></p><h3 style="text-align: justify;">What Should a Report Do?</h3><p style="text-align: justify;">The report should be contextual, compelling, influencing, and targeted to the intended audience to act upon on making a decision. <b><i>The software testing report is not an exemption from it.</i></b></p><div style="text-align: justify;">The performance testing report should know</div><div style="text-align: left;"><ul style="text-align: left;"><li style="text-align: justify;">Who are its audiences?</li><li style="text-align: justify;">How they read, relate and understand the information?</li></ul><div style="text-align: justify;">If this is ignored, the report will not serve the purpose of commissioned testing. <i><b>The effectiveness of a report cannot be determined solely on how the stakeholders responds to it</b></i>. </div></div><p style="text-align: justify;">On understanding the risks and problems in the system's current capability, mentioned in the report, the stakeholder might <b>not </b>respond with an action to tune the performance aspects. This could be for multiple factors including that of business.</p><blockquote><p style="text-align: justify;"><i>Note that, a skilled and problem solving engineer understands the business and how it drives. Just being technically skilled will not help an engineer to grow in a longer run in her or his career. The system's performance tuning decisions most times will be driven by business.</i></p></blockquote><p style="text-align: justify;">Did the report persuade the stakeholders with an awareness, mutual understanding and agreement? <i><b>The report should drive this conversation. If not, we have a problem.</b></i></p><p style="text-align: justify;">On reading the performance testing report, <b style="font-style: italic; text-decoration-line: underline;">do the stakeholders get an informed awareness on what happened during testing in the present capability of a system's performance criteria?</b> <b style="font-style: italic; text-decoration-line: underline;">Do the stakeholders understand and mutually acknowledge how it benefits and costs the business?</b></p><blockquote><p style="text-align: justify;"><i>This is the foremost value serving expected from a testing report. If not, I look at how the data and story is presented in the report.</i></p></blockquote><p style="text-align: justify;">The bottom line is, did we mutually acknowledge, agree and understand on current capability and consequences? If not, the basic purpose of the report is not met.</p><p style="text-align: justify;"><br /></p><h3 style="text-align: justify;">Performance Testing Report</h3><p style="text-align: justify;">The software testing is a high technical activity. You agree or not to this, but, this is the reality.</p><div style="text-align: justify;">Testing for performance is technical investigation activity. It includes the orchestrated study in correlation of </div><div style="text-align: left;"><ul style="text-align: left;"><li style="text-align: justify;">hardware, operating system, network, tech stacks & software used in SDLC, architecture, designs, certain decisions, people, business and you - the test engineer.</li></ul></div><p style="text-align: justify;">The fundamental in-depth awareness and knowledge of these areas is essential and a necessity to analyze the performance's aspect. The performance testing report will show this trait of you as a test engineer.</p><p style="text-align: justify;">We have stakeholders who work in technical area and in non-tech area. How to compile the effective performance testing report?</p><p><i></i></p><blockquote style="text-align: justify;"><i>There is no one way or defined way of writing an effective performance testing report. Figure out what works in context of your testing to have a effective report knowing - <b>What Should a Report Do?</b></i></blockquote><p></p><p style="text-align: justify;"><br /></p><h3 style="text-align: justify;">Outline of Persuading Performance Test Report</h3><div style="text-align: justify;"><br /></div><div style="text-align: justify;">It is a technical story telling in a non-technical way with data, pictures, comparison by relating, metaphors, and contemporary history. I compose the performance testing reports in line with business targets and objectives set. I provide a metaphors to relate and know the value and cost.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">At times, I will have two reports. I share it with respective stakeholders.</div><div><ul style="text-align: left;"><li style="text-align: justify;">One with non-technical summary and conclusion</li><li style="text-align: justify;">The other with technical details, analysis and data from investigation</li></ul><div style="text-align: justify;">Sometimes, I include the above two reports in one report based on the context.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">In overall, this will be in minimum as part of my performance testing report to start.</div></div><div style="text-align: left;"><ol style="text-align: left;"><li style="text-align: justify;">What part of the system is being tested?</li><li style="text-align: justify;">Why that part of the system is being tested?</li><li style="text-align: justify;">Mentioning the vague performance requirements gathered from stakeholders.</li><li style="text-align: justify;">Refining and precising the performance requirements to be specific, contextual and deterministic.</li><li style="text-align: justify;">Who are the stakeholders of this report?</li><ul><li style="text-align: justify;">What sections the respective stakeholders to refer for the analysis and outcome?</li></ul><li style="text-align: justify;">Problem statement of the performance testing statement</li><li style="text-align: justify;"><b>Brief summary of performance testing outcome. </b>[TLDR]</li><ol><li style="text-align: justify;">What aspect of system's performance is evaluated and why?</li><li style="text-align: justify;">Brief summary of performance test carried out and outcome.</li></ol><li style="text-align: justify;"><b>Detailed Report with Technical Details</b></li><ol><li style="text-align: justify;">Analysis and Technical Investigation</li><li style="text-align: justify;">Representation of data which is analyzed</li><li style="text-align: justify;">Identification of bottlenecks, risks, problems and its symptoms</li><li style="text-align: justify;">Summary of the test's outcome</li></ol></ol></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">You don't have to stick on to one format or a template. Figure out what works well in your case so that the intent of your tests and outcome is understood by stakeholders. Give a structure to your report!</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">The performance testing reports will have metrics, graphs, numbers and proposals. <b><i>The presence of metrics, graphs, numbers and the other said, does not make the report effective.</i></b> Then, what makes it effective? When you call it effective? When you accept it is not effective? Only, you can figure it out to your context. I can assist you here; pull me in.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">There is no good [effective] report or not good [ineffective] report. The report is either</div><div style="text-align: left;"><ul style="text-align: left;"><li style="text-align: justify;">From a team with skills, experience, trained, and practicing</li><li style="text-align: justify;">From a team which is not trained, and, not practicing</li></ul><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br /></div></div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-7028399228927965110.post-56539580027436643362024-02-25T03:29:00.001+05:302024-02-25T03:29:37.464+05:30Backtracking of Testing, Security and Tools<p> </p><p style="text-align: justify;">When I started my software testing career in 2006, I was in this thought -- What tools should I use, so that,</p><p></p><ul><li style="text-align: justify;">I can do the testing that is sought after</li><li style="text-align: justify;">I can test for performance</li><li style="text-align: justify;">I can test for security</li></ul><p></p><p style="text-align: justify;">Moving from a search for tools to building the mindset and attitude. <b>It is a journey!</b> It took me time to see this journey. I hopped on to this journey in 2011. I see, this is not an ending journey, while I know where should I go and reach. <b>I'm on this journey</b>.</p><p style="text-align: justify;">I had no mentors. I had no seniors in software testing to guide and discuss on my thought process. I had developers (programmers) who had little or no interest in testing; so it did not matter to them. But, they have helped me to be better tester. I'm grateful to them. Then, the community was not so connected, organized and share the knowledge as it does in 2024. The software testing was not considered or seen as a technical activity, then. I have stood, fought, demonstrated and delivered my testing as a technical activity. I'm continuing it.</p><p></p><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Today, on 24th Feb 2024, I read the below question in a community's social space and decided to write this blog post.</div><blockquote><div style="text-align: justify;"><i>Hey, everyone .... Can anyone please suggest a good tool for API security testing?</i></div></blockquote><p style="text-align: justify;">This question resonates in test engineers. Most of we test engineers still look and ask for tools when it comes to security testing. To test engineers, the performance and security testing are still a conception and activity with tools alone. <b>In reality, it is not!</b> If you are in such thought or you come across such question to answer, this blog post is for you.</p><p style="text-align: justify;"><br /></p><h3 style="text-align: justify;">Backtracking the Problem Identification</h3><p style="text-align: justify;">In programming, we have an approach by name Backtracking. <i>It is about exploring in possible ways to find possible solutions for a problem. And, a best solution which works in context is picked.</i></p><p style="text-align: justify;">What's the problem here? Testing, Security and Tools. Are you with me so far? Let us backtrack this problem.</p><p style="text-align: justify;"><u>Note</u>: <i>I see a difference between the words 'possible' and 'all'. Hence, I use the words "possible ways" and "possible solutions" and not "all ways and all solutions".</i></p><p style="text-align: justify;"><i><br /></i></p><h4 style="text-align: justify;">Bounties and Entry</h4><p style="text-align: justify;">There are reputed bug bounties for security testing. To get into this bounties one has to showcase her/his discoveries and skills with her/his recognized portfolio.</p><p style="text-align: justify;">The tools are accessible to all. The community edition and licensed edition tools are available. We use these both editions of tools.</p><p style="text-align: justify;"></p><ul><li>But, why not all of us with tools cannot get into such invited security bug bounties? </li><ul><li>You will answer this question if you ask yourself. Hope this backtracking should have helped by now!</li></ul></ul><p></p><p style="text-align: justify;">The Security Engineering is a vast practice area in Software Engineering. There are dedicated security engineers in role. But, we test engineers can take up the testing for the security of software systems which the team is programming and building.</p><div style="text-align: justify;">I advise, a practicing test engineer</div><div style="text-align: justify;"><ul><li>To start with building an interest for security engineering.</li><li>Consistently hone and build the mindset, attitude and skills needed for the testing the security aspects.</li><li>Pick simple problems, solve it. Do it consistently, while you explore the layers.</li></ul></div><p style="text-align: justify;">While this is done consistently, it is time to find the mentors in Security Testing. The mentors will assist you in practicing how to test effectively for security making use of simple contextual necessary tools. Also, a mentor will let you know how to test for security without tools to an extent. The tool is effective when known how to use it. The tools help immensely only if I can test for security. </p><p style="text-align: justify;">To backtrack in a different perspective, did any tool that you use, find a P1 security problem [or risk] by itself in its scan? Did your programmers acknowledge to that risk or problem? I will pause with these two question to you.</p><p style="text-align: justify;"><br /></p><p style="text-align: justify;"><br /></p><p style="text-align: justify;">Today, my testing for security is confined to systems that I test. I test for web application, mobile apps, web APIs, and database. I can assist here, if you do the home work and ping me.</p><p style="text-align: justify;"><br /></p><p style="text-align: justify;"><br /></p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-7028399228927965110.post-82410802553533701792024-02-03T20:35:00.004+05:302024-02-05T10:44:37.682+05:30Database: Finding the Tables Having Specified Column Name<p> </p><p>In today's pair testing session with a mentee, we were testing for Database I/O. We were on PostgreSQL. One of the questions a mentee had is,</p><blockquote><p><i>How can I figure out the tables having this column name?</i></p></blockquote><p>Running through every tables and exploring if the column being looked for is present or not, is time consuming. It is not a approach to take as well.</p><p>I went through this when I started the ETL testing practice in 2011.</p><p>Here is the query that works on PostgreSQL to find table names which has specified column name.</p><script src="https://gist.github.com/testingGarage/d85a0a99650ca41ca9b79ec57cc06284.js"></script>
<p><br /></p><p><b>Query</b>:</p><div style="text-align: left;"><span style="color: #2b00fe;">select table_name, column_name<br />from Information_Schema.Columns<br />where table_catalog='<b>database_name</b>' and column_name like '<b>%column_name%</b>'</span></div><p><br />It is a better approach to know the precise column name and using the condition as -- column_name='EmployeeId'.</p><p><br />This query should work on MySQL and MSSQL Server. If not working on MSSQL, need to look into the FROM and WHERE clauses if it is vendor specific.</p><p><br /></p><p><br /></p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-7028399228927965110.post-12910427486193807672024-02-03T18:29:00.004+05:302024-02-05T10:45:47.464+05:30Performance Testing - What to Know Before User Behavior and Traffic Pattern?<p> </p><p>This blog post is in series of 100 Days of Skilled Testing. I see, I do not have to pick every questions asked in this series. I pick and share to which I see, I can add value.</p><p>The twelfth question from the season two of 100 Days of Skilled Testing, is:</p><blockquote><p><i>What strategies do you use to simulate realistic user behavior and traffic patterns when conducting performance tests?</i></p></blockquote><div>The twelfth question asked is vague and it needs to be refined for preciseness to pick it up and continue.</div><div><br /></div><div><br /></div><h3 style="text-align: left;">The Question and the Gap</h3><p>I see the below are missing in the above asked question:</p><p></p><ol style="text-align: left;"><li>What aspect of performance is under evaluation?</li><li>What is the system that is being evaluated for a performance's aspect?</li><li>What part of the system is being evaluated for a performance's aspect?</li><ul><li>Queuing? Messaging? Database I/O? Memory? Space? CPU? Client Performance? Functional Module?</li></ul><li>Who are the users? What are their personas?</li><li>How and where the users are accessing the system?</li><li>What is the context of users accessing this system?</li><li>What is the geo location of users who are accessing this system?</li><li>How long these users are connected by accessing this system?</li><li>Are there any differences among these users in their roles and privileges in accessing this system?</li><li>Can the user access system through multiple interfaces?</li><li>Are you assuming the user is on web browser and mobile apps to access this system?</li><li>Is this system you are referring to, is a software system? Or any other system that is controlled environment like - access door, elevators, etc. ?</li><li>You are asking to simulate the user behavior and traffic pattern. Should I assume, I and you know or agree to any volume of user? And, all these users are here for the same purpose when accessing the system?</li><li>Are you considering any time or at a particular time when talking about the traffic pattern?</li><li>Are there any unrealistic users who is accessing your system? You say 'realistic user'.</li><ul><li>Do you see that bots and non-human are also allowed as a user in your traffic?</li></ul><li>Have you evaluated this earlier in your system?</li><ul><li>If yes, do you have the history and data for user behavior and traffic pattern?</li><li>If you don't have, do you allow to use or have your competitor's user behavior and traffic pattern data? </li></ul><li>What is the tech stack of your system?</li><ul><li>What part of your tech stack, you want to evaluate with this user behavior and traffic pattern?</li></ul><li>What is the architecture of your system?</li><li>What part of your system and its architecture is being evaluated with this user behavior and traffic pattern?</li><li>Are you running this exercise for the first time? If not, where I can refer to previous exercises?</li><li>How the interaction and events are handled from its start to completion?</li><ul><li>What all are needed to complete the transaction in work flow?</li><li>How this transaction can go invalid for lack or incorrect data, state and action?</li></ul><li>What is spike, drop, saturation, expected, unexpected, and average numbers in the traffic coming in?</li><li>What do you understand by traffic? Do you mean number of requests coming in?</li><ul><li>Do you mean the being committed I/O operations?</li><li>Do you mean the response received at the other end?</li><li>What is the definition of 'traffic' in this context?</li></ul><li>What is that you want to study and evaluate by the User Behavior and Traffic Pattern information gathered in this context?</li></ol><div><br /></div><div>Using the above questions, I will get an idea to proceed.</div><div><br /></div><div>I will build a model from information I collect using above asked questions. This model we will used to further in testing for a performance's aspect. The value added to the performance test depends on this model as well. To get a better model in context, it is useful to address the gaps. From here, I start to think further.</div><div><br /></div><div><br /></div><div><br /></div><div>What do you ask and look for when building a model for User Behavior and Traffic Pattern?</div><div><br /></div><div><br /></div><div><br /></div><p></p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-7028399228927965110.post-37398652685635809932024-02-03T14:36:00.001+05:302024-02-05T10:46:14.823+05:30Performance Testing - The Unusual Ignorance in Practice & Culture<p> </p><p>I'm continuing to share my experiences and learning for100 Days of Skilled Testing series. I want to keep it short and as a mini blog posts. If you see, the detailed insights and conversations needed, let us get in touch.</p><p><br /></p><p>The ninth question from season two of 100 Days of Skilled Testing is</p><blockquote><p><i>What are some common mistakes you see people making while doing performance testing? How do they avoid it?</i></p></blockquote><p><br /></p><h3 style="text-align: left;">Mistakes or Ignorance?</h3><p><span>It is mistake when I do an action though I'm aware that it is not right in the context.</span></p><p>I do not want to label what I share in this blog post as mistake. But, <b><i>I call it as ignorance despite having or not having the awareness, and the experience</i></b>.</p><p><span>The ignorance said here is not just tied to the SDLC. It is also tied to the organization's practice and culture that can create problems.</span></p><p>To this blog post's context, I categorize the ignorance in these categories -- Practitioner and Organization.</p><p></p><ol style="text-align: left;"><li><b>Practitioner's ignorance</b></li><ul><li>Not understanding the performance, performance engineering, and performance testing</li></ul><ol><ul><li>When said performance testing, taking it as - "It is load testing"</li><li>No awareness on what is performance and performance engineering</li><ul><li>Going to the tools immediately to solve the problem while not knowing what is the performance problem statement</li></ul><li>Be it web, API, mobile or anything,</li><ul><li>Going to one tool or tools and running tests</li></ul><li>No much thinking on how to design the tests in the performance testing being done</li><li>Ignoring Math and Statistics, and its importance in Performance analysis</li><li>No idea on the system's architecture, and how it works</li><ul><li>Why it is the way it is?</li></ul><li><u>The idea of end-to-end is extended and used in testing for performance and having hard time to understand and interpret the performance data</u></li><ul><li><u>How many end-to-end your tests have identified?</u></li><li><u>Can we test for performance to all these identified and unidentified end-to-end?</u></li></ul><li>Relying on the resource/content in internet and applying or using it in one's context without understanding it</li><li>No idea on the tech stack and how to utilize the testability offered by it in evaluating the performance</li><li>Not using or asking for testability</li><li>Getting hung to most spoken and discussed 2 or 3 tools on the internet</li><li>Applying tools and calling out it as performance testing</li><li>No attempting to understand the infrastructure and resources</li><ul><li>How it impacts and influences the performance evaluation and its data</li></ul><li>Idea on Saturation of resources</li><ul><li>Thinking it as a problem</li><li>Thinking it as not a problem</li></ul><li>Not working to identify where will be the next bottleneck when solving a current bottleneck</li><li>What to measure?</li><li>How to measure?</li><li>When to measure?</li><li>What to look when measuring?</li><li>Not understanding the OS, Hardware resources, Tech Stacks, Libraries, Frameworks, Programming Language, CPU & Cores, Network, Orchestration, and more</li><li>Not knowing the tool and what it offers</li><ul><li>I learn the tool everyday; today, it is not the same to me compared to yesterday</li><ul><li>I discover something new that I was not aware of what it offered and exist</li><li>I learn the new ways of using the tool in different approaches</li></ul></ul><li>No story in the report with information/image that is self-describable to most who reads it</li><li>And, more; but the above said resonates with most of us</li></ul></ol><li><b>Organization's ignorance</b></li><ul><li>At the org level, for first and to start, it is ignorance in Performance Engineering</li></ul><ol><ul><li>Ignoring the practice of performance engineering in what is built and deployed</li><li>Thinking and advocating, increasing the hardware resources will increase and better the performance</li><ul><li>In fact, it will deteriorate over a period of time no matter how much the resources are scaled up and added</li></ul><li>Ignoring the performance evaluation and its presence in CI-CD pipeline</li><li>The performance tests on CI-CD pipeline should not take beyond few minutes</li><ul><li>What is that "few minutes"?</li></ul><li>Not prioritizing the importance of having the requirements for Performance Engineering</li></ul></ol></ol><div><br /></div><div>Recently, I was asked a question - <i>How to evaluate the login performance of a mobile app using a tool "x"?</i></div><div><br /></div><div>In another case, I see, <i>a controller having all HTTP requests made when using web browser. Running these requests and trying to learn the numbers using a tool.</i></div><div><br /></div><div><br /></div><div>I do not say this is wrong way of doing. That is a start.</div><div><br /></div><div>But, we should <b>NOT</b> stay here thinking this is a performance engineering and that is how to run tests for learning a performance aspect[s].</div><div><br /></div><div><br /></div><div>To end,<b> the performance is not just - <i>how [why, when, what, where] fast or slow?</i></b> If that is your definition, you are not wrong! That is a start and good for start; but, do not stick on to it alone and call performance. It is <b>capability</b>. It is about getting what I want in the way I have been promised and I expect; this is contextual, subjective and relative. The capability leads to an experience. What is that experience experienced?</div><div><br /></div><div>Sometimes, serving the requests by what you call as slow, is a performance. What is slow, here?</div><div><br /></div><div>The words fast and slow are subjective, contextual and relative. It is one small part of performance engineering.</div><div><br /></div><div>That said, let me know, what have you been ignoring and unaware in practice of Performance Engineering & Testing?</div><div><br /></div><div><br /></div><p></p>Unknownnoreply@blogger.com3tag:blogger.com,1999:blog-7028399228927965110.post-66966543937931901902024-02-02T00:31:00.003+05:302024-02-02T00:31:55.320+05:30Deep Link and its Testing via Automation<p style="text-align: left;"> </p><p style="text-align: left;">I get these question consistently from my fellow testers and community.</p><p></p><ol style="text-align: left;"><li style="text-align: left;">How to automate the mobile apps and web applications using Deep Links?</li><li style="text-align: left;">How to automate the business flows using Deep Links?</li><li style="text-align: left;">How to achieve end-to-end business flows testing on using Deep Links?</li><li style="text-align: left;">How to automate scenarios in mobile apps using Deep Links?</li><li style="text-align: left;">What is the best approach to automate the mobile apps using Deep Links?</li><li style="text-align: left;">What is the best practice to automate using the Deep Links?</li></ol><div style="text-align: left;">And, more questions on same pitch.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><br /></div><h4 style="text-align: left;"><span style="font-size: large;">No Deep Dive into - What is Deep Link?</span></h4><div style="text-align: left;"><br /></div><div style="text-align: left;">A hyperlink in HTML is a kind of deep link within a website or to another website.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">Deep Link is known with different names for web, Android app and iOS app. All these names have the same understanding and intent at some point.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">The Deep Links are URIs that takes me directly to a specific part (activity or fragment) of the app that I'm using or testing. The Deep Link will have an intent which tells where I will be taken on using it.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">When we converse on diving deep technically into testing and automation of Deep Link, will share more insights into its internals.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><br /></div><div style="text-align: left;"><br /></div><h4 style="text-align: left;"><span style="font-size: large;">Deep Link and Challenges</span></h4><div style="text-align: left;"><span style="font-size: large;"><br /></span></div><div style="text-align: left;">This question is discussed with me often:</div><blockquote><div style="text-align: left;">How to do end-to-end testing using the Deep Link?</div></blockquote><div style="text-align: left;">Automation of a mobile app using Deep Link poses a challenge which is not experienced in web application. </div><div style="text-align: left;"><br /></div><div style="text-align: left;">One such challenge is, say you have <b>not</b> installed the mobile app. [This is solvable!]</div><div style="text-align: left;"><ul style="text-align: left;"><li>On using a Deep Link, I should be taken to Apple Store or Play Store based on the app.</li><li>I have to install the app.</li><ul><li>Post this, in the traditional automation, I should start traversing the business work flows via GUI.</li><li>Is this adding to the flakiness aspect of automation via GUI?</li></ul></ul></div><div style="text-align: left;"><br /></div><div style="text-align: left;">When we talk so much about flakiness and how to avoid (not prevent), should we exercise business workflows when automating using Deep Link? What you are thinking? Let me know!</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><br /></div><div style="text-align: left;"><br /></div><div><h4 style="text-align: left;"><span style="font-size: large;">Scoping of Automation Using Deep Link</span></h4></div><div style="text-align: left;"><br /></div><div style="text-align: left;">Back to the fundamentals.</div><div><ul style="text-align: left;"><li style="text-align: left;">We have to automate, no escape from it. Let us automate what must be automated!</li><li style="text-align: left;">Let us not fall into trap of "Automate everything!"</li><ul><li style="text-align: left;">For today, I'm in this mindset and attitude,</li></ul><li style="text-align: left;">What we automate depends on the objective or goal that we want to accomplish.</li><ul><li style="text-align: left;">Each test should have precise and deterministic goal.</li><ul><li style="text-align: left;">A test via automation is not an exemption to it.</li><li style="text-align: left;">A test defined in automation should be precise, deterministic and have a single objective - Single Responsibility Principle.</li></ul></ul></ul><div style="text-align: left;"><br /></div></div><div style="text-align: left;"><b><i>What is the objective of my testing via automation for the Deep Link?</i></b> This define the scope and extent of my automation. This will minimize the number of checks that I do using Deep Link.</div><div><div style="text-align: left;"><br /></div></div><div style="text-align: left;">The purpose of Deep Link is to take me to specific part of the mobile app.</div><div style="text-align: left;"><ul style="text-align: left;"><li>Should I start the end-to-end or exercising the workflow to be included in the Deep Link tests?</li><ul><li>If included, am I not complicating the testing via automation?</li></ul></ul></div><div style="text-align: left;"><br /></div><div style="text-align: left;"><br /></div><div style="text-align: left;"><br /></div><div style="text-align: left;"><span style="font-size: large;"><b>Automation using Deep Link</b></span></div><div style="text-align: left;"><br /></div><div style="text-align: left;">I ask this question to myself and to my team.</div><blockquote><div style="text-align: left;">What is the goal of testing via automation using Deep Link?</div></blockquote><p style="text-align: left;">This question helps me to pick minimal and necessity flow actions. It has lead and leads me to define minimal tests for Deep Link based on what we want to learn from automation of same.</p><p style="text-align: left;">To me, <b><i>the purpose of Deep Link is not end-to-end testing</i></b>. It's purpose is,</p><blockquote><p style="text-align: left;">Am I taken to the intended state and data when used the Deep Link?</p></blockquote><p style="text-align: left;">I have kept the test intent to this.</p><p style="text-align: left;">With this, I have come with tests that has minimal must evaluation and assertion to learn if the app is responding or not to the Deep Link. This is what the business wants when the Deep Links are created.</p><p style="text-align: left;"><b><i>The app usage and workflow function is not a problem statement of Deep Link in a general context</i></b>.</p><p style="text-align: left;">Deep Link is not for end-to-end. It is to take to you from a point to another point, that's it.</p><p style="text-align: left;"><br /></p><p style="text-align: left;">Are you automating using Deep Link?</p><p style="text-align: left;"><br /></p><p style="text-align: left;"><br /></p><p></p>Unknownnoreply@blogger.com3tag:blogger.com,1999:blog-7028399228927965110.post-42995607913284292422024-01-22T12:32:00.000+05:302024-01-22T12:32:35.275+05:30RAAMA: My Test Discovery Model<p> </p><h3 style="clear: both; text-align: justify;">RAAMA -- I Look at You Everyday!</h3><div style="text-align: justify;"><br /></div><div style="text-align: justify;">I have tried to put up one of my Test Discovery models in a conceptual way here with name RAAMA - <i>Refer to, Arrange, Action, Monitor, and Assert</i>.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Maybe this model helps you and your test engineering team as it is helping me. Use this to your context with addition or subtraction for what you are seeking.</div><div style="text-align: justify;"><br /></div><div><span style="text-align: justify;">I refer to this RAAMA of me everyday and when I'm testing. I'm finding the new learning and realization everyday that I was unaware earlier. My understanding of RAAMA is not same what I had on the previous day.</span></div><div class="separator" style="clear: both; text-align: justify;"><br /></div><div class="separator" style="clear: both; text-align: justify;">My understanding of this RAAMA is incomplete and I have made PeACE with it by accepting it. My understanding is growing and getting better everyday. I will share a better version of it as I experience it.</div><div class="separator" style="clear: both; text-align: justify;"><br /></div><div class="separator" style="clear: both; text-align: justify;">Each time I look up to RAAMA and refer to it, I see a new dimension to RAAMA. The awareness, exposure, and the questions are getting better giving the better realization of what I was ignorant and unaware. The RAAMA is exposing me to be a better test engineer today than what I was earlier.</div><div class="separator" style="clear: both; text-align: justify;"><br /></div><div class="separator" style="clear: both; text-align: justify;"><br /></div><div class="separator" style="clear: both; text-align: justify;"><br /></div><div class="separator" style="clear: both; text-align: justify;">RAAMA - I Look at You Everyday!</div><div class="separator" style="clear: both; text-align: justify;"><br /></div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8Xgyxzagn9CuAPwuzHsgFjhKYAeIPyThrBUSuQ1eTBASFA8jf2FCHeQZP6TROzOxzLn3pqShFWOocZU8QMwPaQYPW71tSDoViHkjcdYRWbedyeQSFfzhPcThzgswQ74jOfDIdsyLjG94FdyWcIrK-_lXsA5Isi8Nt0EgHroAkDDkWigRb9tEs9REd1Lg/s2308/RAAMA_testingGarage_v1dot5.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="2308" data-original-width="1545" height="945" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8Xgyxzagn9CuAPwuzHsgFjhKYAeIPyThrBUSuQ1eTBASFA8jf2FCHeQZP6TROzOxzLn3pqShFWOocZU8QMwPaQYPW71tSDoViHkjcdYRWbedyeQSFfzhPcThzgswQ74jOfDIdsyLjG94FdyWcIrK-_lXsA5Isi8Nt0EgHroAkDDkWigRb9tEs9REd1Lg/w632-h945/RAAMA_testingGarage_v1dot5.png" width="632" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;">RAAMA - One of my evolving models for Test Discovery</div><p style="text-align: center;"><br /></p><p style="text-align: justify;">Note: I have not explained in detail what I mean for each node and its sub-nodes. I can talk and discuss it with you if you look for it; I'm just one email away to get started.</p><p style="text-align: justify;"><br /></p><p style="text-align: justify;"><br /></p>Unknownnoreply@blogger.com2tag:blogger.com,1999:blog-7028399228927965110.post-72048219208727310952023-12-03T22:16:00.000+05:302023-12-03T22:16:07.095+05:30A Test Is Not a Metric<p> </p><p>A test execution by human or automation will provide information to be aware & learn.</p><p>If there is a metric, it should be for what I got to be aware of & learned. Not for the number of tests. If it is for a test, it is blunder, before taking it as a metric.</p><p>One can identify infinite tests; automate adding annotation @test in big numbers. Should this number be a metric?</p><p>That is a question to ask when you see a metric on this number.</p><p><br /></p><p>What is of value to you in this outcome from a test? That value should have a metric, and not test or the number of tests.</p><p><br /></p><p>Identifying the metric in a context which serves, is not easy.</p><p>Number of pass/fail or green/red is a measurement; not a metric!</p><p>Anything measured cannot be a metric.</p><p><br /></p><p>A metric helps to measure in a way that establishes rational correlation & upholds its necessity in business.</p><p><br /></p><p>Share this awareness!</p><p><br /></p><p><br /></p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-7028399228927965110.post-54457116538000310932023-11-28T09:59:00.004+05:302024-02-05T10:46:42.657+05:30Behind the Every Test Data, There is a ?!<p style="text-align: justify;"> </p><p style="text-align: justify;">Read <a href="https://testinggarage.blogspot.com/2023/11/test-data-is-not-a-test-and-understanding-test-data-management-ravisuriya.html" target="_blank">this</a> blog post to have a perspective about the Test Data and Test Data Management. The point is, if I'm not aware of a test and what does it tell me to explore, I cannot think of a Test Data.</p><p style="text-align: justify;"><span>That said, </span><i><b>if I know what I should be evaluating as part of performance, why, when and how, this will help me to come up with a thought for identifying the tests and its test data for the same</b></i><span>.</span><span style="text-align: left;"> </span></p><p style="-webkit-text-stroke-width: 0px; text-align: justify;">The ninth question from season two of 100 Days of Skilled Testing is:</p><blockquote><p style="-webkit-text-stroke-width: 0px; text-align: justify;">What role does data management play in performance testing, and how do you ensure the availability of suitable test data sets?</p></blockquote><p style="text-align: justify;"><br /></p><h3 style="text-align: justify;">Testing and "Ensure"</h3><p style="text-align: justify;">We test and have tests in testing, because, there is no "sure" and "ensure" idea in software. But, we presume on a rational basis upon, "if, these are this", in a given context when the software processes.</p><p style="text-align: justify;">Now, ask yourself, how can we ensure the availability of suitable test data sets?</p><p style="text-align: justify;"><i>In my opinion, <b>the Test Data is often misunderstood</b>. This is the primary problem and should be the first problem, when asked "what are the challenges in creating the test data?".</i></p><p style="text-align: justify;">When you read the concluding lines of this blog post, you will learn why I say this.</p><p style="text-align: justify;"><br /></p><h3 style="text-align: justify;">Test Data and Immunity</h3><p style="text-align: justify;">In my opinion and experience in practicing the Test Engineering, I see, <b>the Test Data should be a viral strain and it should have its variants</b>. When this test data is used to test [experiment, test investigate, and debug], how do the software and its ecosystem respond?</p><p></p><ul style="text-align: left;"><li style="text-align: justify;"><b>Does the software and its ecosystem is immune to this test data?</b></li><ul><li style="text-align: justify;"><b>Does it exhibit any risks and problems?</b></li><ul><li style="text-align: left;">If yes, then, do the purpose of my testing and automation is accomplished with this test data?</li></ul></ul></ul><div style="text-align: justify;">This puts me back to question, <b><i>what is the purpose [intent] of my test?</i></b> It drives me to derive the test data which helps me to know -- <i>What am I supposed to learn and on priority? With this, </i>I get an idea for what kind of test data I should be creating knowing its pattern.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">If the system is immune to Test Data and not reveling anything new in the context, I classify this pattern of test data as "Immune" to the context.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">In my practice and research work in Test Engineering and Software Testing, to start, <b>I categorize Test Data into two areas</b>.</div><div><ol style="text-align: left;"><li style="text-align: justify;"><b>Immune</b></li><li style="text-align: justify;"><b>Not Immune</b></li></ol><div style="text-align: justify;">Further, I have categories, under these two, where I classify the Test Data deterministically for the context. Get in touch if you want to learn more about this. I'm just one ping away!</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">The tests should help me to evaluate for the immunity and also non-immunity; both are essential and necessity. </div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">The <b>credit</b> is to me for such classification of Test Data. It is my research work out of my practice.</div></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Note that, Test Data is not just the input [characters or files] entered or given to a system. Test Data has its association to tech stacks, infrastructure, ecosystem, business workflows and people. To craft such Test Data, one has to have the understanding of the system and its internals, and, the problem it solves by knowing how it solves.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br /></div><h3 style="text-align: justify;">Performance Testing and Test Data</h3><div style="text-align: justify;"><ol><li style="text-align: left;">What is that I'm testing as part of performance?</li><li style="text-align: left;">What do I want to evaluate in the name of performance?</li><li style="text-align: left;">What part of the system is evaluated for its performance?</li><ul><li style="text-align: left;">Should I evaluate this in isolation or as a wholeness of the system?</li></ul><li style="text-align: left;">What domain knowledge and information I should have when testing for performance?</li><li style="text-align: left;">What system's architecture and internal details I should understand and be aware to test for performance?</li><li style="text-align: left;">Is this the first delivery? Or, do we have this system running in the production?</li><ul><li style="text-align: left;">If it is first delivery,</li><ul><li style="text-align: left;">How will I create the test data to suit the consumers of this application?</li><li style="text-align: left;">What are the key workflows of business that we should be evaluating for its performance?</li><li style="text-align: left;">Do all workflows and sub-systems need the evaluation for performance, and on priority?</li><li style="text-align: left;">How do I map the fragmentation of users and their data [with its patterns]?</li><li style="text-align: left;">What are the infrastructure and ecosystem characteristics that should be part of the test data identified?</li><li style="text-align: left;">Does caching have any effect if the same pattern of data is used?</li></ul><li style="text-align: left;">If it is a running version in production</li><ul><li style="text-align: left;">Can I refer to the DB to figure out the pattern for the particular workflow that I'm evaluating?</li><li style="text-align: left;">How can I match the test data to have the production data's characteristics and attributes?</li></ul></ul><li style="text-align: left;">What is the backup strategy for the Test Data?</li><ul><li>How do I version control the Test Data?</li><li>Which version of the Test Data I should be using?</li></ul><li style="text-align: left;">What is the threshold I'm targeting with Test Data?</li><ul><li style="text-align: left;">What should be the size of the data in DB when I make the IO and RW operations?</li><li style="text-align: left;">What should be the network capability when I make the IO and RW operations?</li><li style="text-align: left;">What should be the hardware capability when I make the IO and RW operations?</li><li style="text-align: left;">What should be the geographical traffic and its pattern when I make the IO and RW operations?</li><li style="text-align: left;">More of such factors will be considered when identifying and deriving the test data.</li></ul><li style="text-align: left;">What is the client error yielding Test Data that I should have for the workflow?</li><li style="text-align: left;">What is the server error yielding Test Data that I should have for the workflow?</li><li style="text-align: left;">What is the redirection yielding Test Data that I should have for the workflow?</li><li style="text-align: left;">What is the no-response and no-change Test Data that I should have for the workflow?</li></ol><div>And, more. It is simple; get in touch to discuss and know beyond the listed.</div><div><br /></div><div><br /></div><div><br /></div><div>To conclude and stop here, all these questions, do not ensure or assure or make sure that I will have test data for evaluating a characteristic of performance.</div><div><ul><li>It helps me to know:</li><ul><li>What are the tests I should be doing?</li><li>What kind of preparation I should be having in my practice to create the Test Data for these tests?</li></ul></ul></div><div><br /></div><div>The, <b>Test Data should challenge the available Testability and its limits</b>. If it is not doing, then, we are having a test data no doubt about it; but, it is of shallow. Shallow!?</div><div><br /></div><div>One has to ask self, "Is this sufficient enough and effective Test Data for the system [and workflow] I'm testing?"</div><div><br /></div><div>The, <b>Test Data should drive the engineering team to add more layers of Testability into the system</b>.</div><div><br /></div><div><br /></div><div><br /></div><div><br /></div></div><p></p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-7028399228927965110.post-32610833030720527312023-11-24T10:29:00.002+05:302023-11-24T10:34:42.848+05:30Test Data is Not [Equal To] a Test<p> </p><p style="text-align: justify;">Not every release is a critical release!</p><p style="text-align: justify;">There are releases which are critical from technical and business interests. In such critical releases, most of my time is spent on understanding the technicalities of the system and the test data identification on identifying what are the priority tests. Identify and building the test data takes major chunk of the testing time. This effort has helped me, testing, stakeholders, projects and products immensely.</p><p></p><p style="-webkit-text-stroke-width: 0px; color: black; font-family: "Times New Roman"; font-size: medium; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration-color: initial; text-decoration-style: initial; text-decoration-thickness: initial; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;"></p><blockquote style="-webkit-text-stroke-width: 0px; color: black; font-family: "Times New Roman"; font-size: medium; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration-color: initial; text-decoration-style: initial; text-decoration-thickness: initial; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;"></blockquote><p></p><p style="-webkit-text-stroke-width: 0px; color: black; font-family: "Times New Roman"; font-size: medium; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: justify; text-decoration-color: initial; text-decoration-style: initial; text-decoration-thickness: initial; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;"><br /></p><h3 style="text-align: left;">Test Data != Test</h3><p style="text-align: justify;">Looks like the word "Test Data Management" has become a buzz and marketing word. In how the word "Test Data Management" is pushed, to me it looks like the intent is missing for -- <i>how to identify the tests and its test data?</i></p><p></p><div style="text-align: justify;"><ul><li>If I cannot think of tests, how can I think of Test Data?</li><ul><li>How can I think of tests, the right tests in the context, and the priority tests out of it? </li><ul><li>If I cannot get this, how can I get the Test Data for all these tests?</li></ul></ul></ul></div><p></p><p style="text-align: justify;">Before talking of Test Data Management, we need to talk about the intent and goal of the test. If I have a clarity here, by being aware of the product's technical internals, architecture and its eco system, I can think of test data in a better sense. For first, the PaaS which provides Test Data Management solution has to push and enforce for learning and mentioning -- What is the intent of the test?</p><p style="text-align: justify;">When I know, for what to evaluate and how should I be evaluating it, those test intent will lead me to the Test Data and its dimensions which should be vectoring the tests. </p><p style="text-align: justify;">Test Data is not [equal to] a Test. Test Data is one of the variance, in fact, it is a multi-dimensioned variance that a test will [need to] have. While a test has one deterministic objective, the test data will have multi-dimensioned vectors that instruments a test.</p><p style="text-align: justify;">Test Data has its role in a test, and, it is a critical role. Hence, we practicing Test Engineers talk and spend our time on Test Data in equal importance as we give our time to identifying, designing and execution of tests.</p><p style="text-align: justify;"><i style="font-weight: bold;">Test Data is not a test; but the test data is an expression of the test's intent.</i> Test Data is one of the byproducts of a test. Having this clarity is important.</p><p style="text-align: justify;">Note this, the test data is not the only expression of the test's intent. <b><i>A test will have multiple touch points; these touch points express, advocate and can show the intent of a test</i></b>. A test data is also a heuristic as a test is.</p><p style="text-align: justify;"> </p><h3 style="text-align: justify;">Test Data Management</h3><p style="text-align: justify;">The word "management" is underrated and not attempted to understand from context where it is being spoken. How do this word sound -- "<b>Test Data Leadership</b>"?</p><p style="text-align: justify;">In engineering, especially in the software engineering, the word "management" inherently talks primarily about design. On day-to-day operation, the management designs its strategy and approach to solve the problem and challenges. </p><p style="text-align: justify;">When said "Test Data Management" in software engineering, it is about strategizing and approaching the problem of identifying and categorizing (subsets) the data to test. A kind of leadership at one layer. It has its role and critical to the deterministic outcome for a test.</p><p style="text-align: justify;">In machine learning, we can see this categorization -- training data and test data. Then, is training data not a test data? It is! The training data is designed [managed] to train and this data as well identifies the problem as the training progresses.</p><p style="text-align: justify;">The data which are identified, designed, categorized and collected as Test Data, are well sampled data in the context, for the intent of a test.</p><p style="text-align: justify;">To conclude, Test Data Management is one of the correlation in software engineering to solve a problem; it is contextual in how and what tests and testing is executed. The different teams and organizations has their own way of managing the test data. How they do it, it is their way of doing it. We have to look Test Data Management from point of Test Engineering and not as an another buzz word to sell just as a product and business's service.</p><p style="text-align: justify;"><br /></p><p style="text-align: justify;"><br /></p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-7028399228927965110.post-35442748144465386542023-11-19T23:00:00.002+05:302024-02-05T10:47:42.350+05:30Waterfall or Agile: Testing for Performance - Where to Start?<p> </p><p style="text-align: justify;">Do you understand the Agile? I have shared my understanding <a href="https://testinggarage.blogspot.com/2023/11/the-agile-testing-how-shift-left-shift-right-are-important-in-continuous-delivery-ravisuriya.html" target="_blank">here</a>; give it a read.</p><p style="text-align: justify;">The eighth question from season two of 100 Days of Skilled Testing is:</p><p></p><blockquote><p style="text-align: justify;"><i>Can you share some best practices for conducting performance tests within an Agile development environment?</i></p><div></div></blockquote><p style="text-align: justify;"><br /></p><h3 style="text-align: justify;"><span style="text-align: left;">Best Practices and the Agile</span></h3><div style="text-align: justify;"><div style="text-align: justify;"><br /></div><div style="text-align: justify;">The irony is, the Agile says, <b>there is no best practice</b>. It asks, to tailor and fit the practice to the context so the continuous delivery and value is delivered consistently upholding the Agile's principles. </div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Yet, we talk about the best practices in the Agile's context, like the eighth question asked here.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">What is the effective way to test in the continuous delivery?</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">As a test engineer, how can I start thinking and testing for performance from the inception of a feature's thought? I see, it is not hard to do so. As you read further in this post, you will have a perspective and awareness to do it.</div></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><h3 style="text-align: left;"><span style="text-align: justify;">Performance in Waterfall and Agile</span></h3></div><div><p style="text-align: justify;">I learn, the <a href="https://testinggarage.blogspot.com/2023/11/the-performance-is-a-perception-and-different-facets-of-performance-in-software-ravisuriya.html" target="_blank">performance</a> is an experience. It does not differ because of the Waterfall or Agile. If the performance is not a pleasing experience, it will impact stakeholders no matter it is Waterfall or Agile.</p><div style="text-align: justify;">But, the question when evaluating for the performance is -- <b><i>where to start, when to start, how to start, and with what to start?</i></b></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">As of today, I do not see differences in the mindset and skills one has to have for testing of performance in Waterfall and Agile. Could be the approach differs in certain phases here; otherwise, I see the same in both practices.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">I will rephrase the eighth question to this:</div><blockquote><div style="text-align: justify;"><b><i>What is your practice to evaluate the performance right from the start of product development in your project?</i></b></div></blockquote><div style="text-align: justify;">I do not want to wait until to hear -- <i>the development is completed and deployed; now we can start running the performance tests</i>.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">What can I do as part of performance tests from the first day of development and first commit? This is my intent and area to look in strategizing the testing and tests.<br /><h3 style="text-align: left;"><br /></h3><div><br /></div><h3 style="text-align: left;">The Culture of Engineering</h3><p style="text-align: left;"><span style="font-weight: normal;">At the start and end of the day, when we <a href="https://testinggarage.blogspot.com/2023/11/who-is-a-developer-in-software-engineering-ravisuriya.html" target="_blank">developers</a> start and finishes the work,</span></p><div><ul style="text-align: left;"><li>How the work is done and why, is defined by the engineering culture practiced by that organization.</li><ul><li>The Performance Engineering of the software products and solution being built will be driven the by the culture practiced.</li></ul></ul><div><br />The Test Engineering and how we test and automate will be driven by the culture of engineering practiced in the organization.</div></div><div><br /></div><div>Writing the code not just for building the functionality, but, also for performance is a culture driven factor. The organization's culture for engineering practice drives it!</div></div><h3 style="text-align: justify;"><br /><br /></h3><h3 style="text-align: left;">Testing for Performance - Where to Start?</h3><div><br /></div><div style="text-align: justify;"><u>I'm sharing my research work that I'm doing and experimenting on performance engineering and performance tests</u>. I'm seeing the results and value out of it and so are the stakeholders.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Today, we are getting skilled in exploring and testing without the requirement document and SLAs in hand. Isn't it? Haven't you?</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">I use my <a href="https://testinggarage.blogspot.com/2023/11/the-minimum-viable-questioning-tests-to-test-and-automate-mvp-mvqt-ravisuriya.html" target="_blank">MVPT</a> to figure out what are the minimum performance tests for the feature. As part of this, I will explore with help available aids to evaluate the performance.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">To start, I will use these questions to figure out the performance tests:</div><div><ul style="background-color: white; font-family: "Open Sans", Georgia, Utopia, "Palatino Linotype", Palatino, serif; font-size: 15.4px; line-height: 1.4; margin: 0.5em 0px; padding: 0px 2.5em;"><li style="margin: 0px 0px 0.25em; padding: 0px; text-align: justify;"><i>What is the minimum viable questioning performance tests that you have got to test this feature?</i></li><li style="margin: 0px 0px 0.25em; padding: 0px; text-align: justify;"><i>What is the minimum viable questioning performance tests that you have got to test this workflow?</i></li></ul></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br /></div><h4 style="text-align: left;">Unit Tests for Time and Space Complexity</h4><div><br /></div><div>I will work closely with programmers to gather information on below when the code for the feature is committed as part of <b>Unit Tests.</b></div><div><ul style="text-align: left;"><li><b>The execution time taken by the code of that feature - the Big O Notations for space and time complexity</b></li><ul><li>Usually the Unit Tests focuses on functional tests and clean code practice</li><li>But, when we test team ask and push for performance data, this can come as part of Unit Tests</li><ul><li>An architect or a principal engineer can set an expectation on</li><ul><li>What should be the time and space complexity of a code for a feature?</li><ul><li>Each functions and blocks need to be evaluated on this</li><li>As said earlier, this depends on a engineering practice culture of an organization</li><ul><li>If the culture wants it, it will be there; else just the functional code will be delivered and not the performance code</li></ul></ul></ul><li>If the time and space complexity analysis outcome is not as expected, the code written has to rethought and refactored</li><ul><li>The review process need to put it back</li><li>The comment with data has to be published</li><ul><li>This will be useful to model the performance tests by test engineers who will be working on performance tests</li></ul></ul><li>Doesn't it look a like a effective useful practice as part of Performance Engineering right in the early stage?</li><ul><li>This is very well applicable to projects running on Agile or Waterfall</li></ul></ul></ul></ul><div><br />Do you have this in your project and Unit Tests written?</div><div><br /></div><div>The time and space complexity questions should not be confined just to the SDETs [test engineer] interview. A test engineer has to ask for it and apply it in her or his day-to-day work.</div><div><br /></div><div><br /></div><h4 style="text-align: left;">Profiling Tests by Test Engineers</h4><div><br /></div><div style="text-align: justify;">We testers do not get into product's code analysis. We have to build skill to run the profiling on product's code and analyze the resources data.</div><ul style="text-align: left;"><li style="text-align: justify;">Test Engineers can test the feature's code with the help of IDE's profiling (<b>runtime analysis</b>) and collect the performance data by identifying the performance bottlenecks</li><ul><li style="text-align: justify;">This runtime analysis can profile for</li><ul><li style="text-align: justify;">Memory snapshots</li><li style="text-align: justify;">Thread analysis</li><li style="text-align: justify;">Monitoring resources</li><li style="text-align: justify;">CPU and allocation profiling</li><li style="text-align: justify;">And, more</li><li style="text-align: justify;">The problems and risks can be reported upon analysis</li></ul><li style="text-align: justify;">Compare the two different solution's approach performance data</li></ul></ul></div><div style="text-align: justify;">This information will tell and indicate where is the risk and problem when we deploy the code. In my opinion, this is a useful information in modeling further performance tests. This information is first-hand information which is very powerful before we start using any other performance testing strategies and tools to aid the tests.</div><div><br /></div><div><br /></div><div><br /></div><h3 style="text-align: left;">Get Started with Performance Engineering and Tests</h3><div><br /></div><div style="text-align: justify;">These are available in the IDE. We think of performance testing tools and ask how to test for performance. To be precise, we test developers (test engineers) should change our mind and shift for first. If not, as I say, we will be the bottleneck for first to ourselves. Did you know this way of testing for performance? Why not you introduce this in your project and organization?</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">If seen, these test practices can be used right from the day we commit the feature's code. This is a place to start for the performance tests. This will be a differentiator together with <a href="https://testinggarage.blogspot.com/2023/11/the-minimum-viable-questioning-tests-to-test-and-automate-mvp-mvqt-ravisuriya.html" target="_blank">MVPT</a> and guides the MVPT to design effective performance tests in the context.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">I do not say these are best practices and there is no best practices. But this is a useful practice when the organization and stakeholders ask for it. Let your organization and stakeholders know how well you can test for performance right from first commit of product's code.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">To stop and end here,</div><div style="text-align: justify;"><ol><li style="text-align: left;">Just do not test for functionality from day one, also test for the performance from the day one.</li><li style="text-align: left;">Influence your organization's engineering culture and <a href="https://testinggarage.blogspot.com/2023/11/who-is-a-developer-in-software-engineering-ravisuriya.html" target="_blank">developers</a> not just for developing functional code, but, also for the performance code</li></ol></div><div style="text-align: left;"><br /></div><div><br /></div><div><br /></div><div><br /></div><p></p></div>Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-7028399228927965110.post-86431686910865726062023-11-19T16:33:00.004+05:302024-02-05T10:48:08.857+05:30Is Performance a Perception to an Engineer and User?<p> </p><p style="text-align: justify;">When hearing about performance from customers and engineers on team, I see each are having a perception of it on using a product. To one it was fast enough, to another it was as usual and for one it was too slow. Each are expressing their perception. But, what is the performance?</p><p><br /></p><h3 style="text-align: justify;">What is Performance?</h3><div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">I see, understanding and knowing "<u>What is performance?</u>" is important for everyone who is involved in building the software system and product. In my opinion, this should be the start point. It is beneficial, when everyone has a shared common understanding to it as a team and business.</div></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Performance is an <b>emotion</b> to a user! Technically, the performance has multiple facets to it for understanding the <b>capability</b> of a system and its sub-systems.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br /></div><h4 style="text-align: justify;">Facets of the Performance</h4><div style="text-align: justify;"><br /></div><div style="text-align: justify;">What facet do you consider and call it out as Performance? This is another point where most of us do not align. For example, below are the different facets that we usually hear and read a lot:</div><div><ul><li style="text-align: justify;">Heap and Memory</li><ul><li style="text-align: justify;">Threads</li><ul><li style="text-align: justify;">Not supporting for concurrency</li><li style="text-align: justify;">Not handled well in concurrency</li><li style="text-align: justify;">Holding up other processes and causing bottleneck cases</li></ul><li style="text-align: justify;">Memory Leaks</li><li style="text-align: justify;">Concurrency</li><li style="text-align: justify;">Memory not reclaimed</li><li style="text-align: justify;">and, more ....</li></ul><li style="text-align: justify;">CPU consumption</li><ul><li style="text-align: justify;">Open connections and its I/O</li><ul><li style="text-align: justify;">Held up in processing requests</li></ul><li style="text-align: justify;">No enough resources to process</li><li style="text-align: justify;">and, more ....</li></ul><li style="text-align: justify;">DB I/O</li><ul><li style="text-align: justify;">Open connections</li><li style="text-align: justify;">Unindexed data</li><li style="text-align: justify;">SPs and Queries holding the transactions</li><li style="text-align: justify;">Incorrect configurations of DB Server and nodes</li><li style="text-align: justify;">and, more ...</li></ul><li style="text-align: justify;">Disk I/O</li><ul><li style="text-align: justify;">Running out of space</li><li style="text-align: justify;">RW I/O not responsive</li></ul><li style="text-align: justify;">Network consumption</li><ul><li style="text-align: justify;">Unmanageable transactions</li><li style="text-align: justify;">Transaction's data, size and time</li></ul><li style="text-align: justify;">Latency</li><ul><li style="text-align: justify;">Latency in which interfaces?</li><li style="text-align: justify;">Ineffective caching, queuing and messaging</li></ul><li style="text-align: justify;">GUI not rendered or painted</li><ul><li style="text-align: justify;">GUI exhibiting jank behavior</li><li style="text-align: justify;">GUI partially rendered</li><li style="text-align: justify;">GUI not in a interactive state</li><li style="text-align: justify;">GUI not responding to an action</li><li style="text-align: justify;">GUI and UI loading multiple times with no room for interaction </li></ul><li style="text-align: justify;">Terminal yet to return and show the prompt</li><li style="text-align: justify;">Older and deprecated libraries with latest dependencies</li><li style="text-align: justify;">Server, orchestration, and sub-systems configured incorrectly</li><li style="text-align: justify;">Hardware resources and its specifications used in a context</li><li style="text-align: justify;">Display refreshing rate and frames lost with GUI rendered</li><li style="text-align: justify;">Heat dissipation</li><ul><li style="text-align: justify;">From the hardware</li><li style="text-align: justify;">Experienced in the environment</li></ul><li style="text-align: justify;">Time taken for a request to reach the actual end point</li><li style="text-align: justify;">Execution time on receiving a request</li><li style="text-align: justify;">And, more...</li></ul><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Further we have classified it to frontend and backend; both are important and equally needed. The webpage has got different KPIs and metrics to determine where do its performance stand. Likewise, for backend.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Which facet of performance need to be evaluated and in which phase? Why? The perception will be established when testing, on how we test it, and how one uses the product.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><b>With all these for performance, where to start and what to look at?</b> This is one of the question with which we are left in Waterfall and Agile. That is where the eighth question of 100 Days in Skilled Testing comes in -- <i>Can you share some best practices for conducting performance tests within an Agile development environment?</i></div></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Have you asked these questions to yourself and team?</div><div style="text-align: justify;"><ol><li>What is performance to you and to your stakeholders?</li><li>What should I consider in evaluating for the performance of your software system?</li><li>What is the practice I want to pick to evaluate the performance?</li><li>Where do I start and how? </li></ol></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br /></div>Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-7028399228927965110.post-83222193730104460242023-11-19T02:31:00.000+05:302023-11-19T02:31:23.399+05:30MVQT: The Testing and Tests with a MVP's Perspectives<p style="text-align: justify;"><br /></p><p style="text-align: justify;">I was leading multiple teams and its delivery in a testing service company. Then, I came up with this thought -- <i><b>Like MVP, I also have the MVT (Minimum Viable Tests) for a MVP</b></i>.</p><p style="text-align: justify;">Further, I expanded this thought in my day-to-day practice on tailoring to different contexts. I'm observing that it is applying well to the different contexts when I tailor it to the contexts. After experimenting it for 10 years, I'm sharing this as a blog post.</p><p style="text-align: justify;"><br /></p><h3 style="text-align: justify;">What is a MVP?</h3><p style="text-align: justify;">I take this from Eric Ries. It looks simple and precise to me.</p><p></p><blockquote><p style="text-align: justify;"><i><b>The Minimum Viable Product is that version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort.</b></i></p><p></p></blockquote><p style="text-align: justify;">I see this technique [and a concept] can be applied to anything when I'm developing. As a test engineer, I develop the tests and test code in major as part of my testing. On applying the idea of MVP to my testing and deliveries, I see the value and result.</p><p style="text-align: justify;">Reading this blog post of me to know who is a <a href="https://testinggarage.blogspot.com/2023/11/who-is-a-developer-in-software-engineering-ravisuriya.html" target="_blank">developer</a>.</p><p><br /></p><h3 style="text-align: left;">Testing, Tests, MVP and MVQT</h3><p>In software test engineering, I see the MVP as <u><b>Minimum Viable Questioning Tests</b></u>.</p><p><br />The Minimum Viable 'Q' Tests (<b>MVQT</b>) for a focused area of a feature [or to a feature]</p><p style="text-align: left;"></p><ul style="text-align: left;"><li>Helps me to identify the priority tests that should be executed for first</li><li>Allows me to learn information on priority which matters critically to product and stakeholders</li><ul><li>So that a informed decision can be made.</li></ul></ul><p></p><p><br />The <b>Q</b> in MVQT stands for "<u><b>questioning</b></u>". I read it as <b>Minimum Viable Questioning Tests. </b>I see the "Q" as a placeholder for the Quality Criteria. That is, <u>MVFT</u> means <b><i>Minimum Viable <u>Questioning Functional</u> Tests </i></b>to a feature or a workflow.</p><p><br /></p><div class="separator" style="clear: both; text-align: left;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZJoxLqElpfN2cwUSDlkZtzqPHrcCtY8TAcjqM5x3piAQr0a45w3BGnn1CmWOD9rKtv-_w9yXYxaP1XwaD8C9JMBNqv41kNt5iqIOm0VVYRrMU9mdGssTY2_tQRrMs2PN6l7kq4k65s6XM0J2TiRh1EhO46zxKTPpfXf3m6EGO0V8mpBsEcKlsn0_ZcLY/s2344/IMG_20231119_015951.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1704" data-original-width="2344" height="466" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZJoxLqElpfN2cwUSDlkZtzqPHrcCtY8TAcjqM5x3piAQr0a45w3BGnn1CmWOD9rKtv-_w9yXYxaP1XwaD8C9JMBNqv41kNt5iqIOm0VVYRrMU9mdGssTY2_tQRrMs2PN6l7kq4k65s6XM0J2TiRh1EhO46zxKTPpfXf3m6EGO0V8mpBsEcKlsn0_ZcLY/w640-h466/IMG_20231119_015951.png" width="640" /></a></div><br /><p><br /></p><p>The MVQT are key to know:</p><p></p><ul style="text-align: left;"><li>Have I identified and designed the priority tests? How do I know that I have got them?</li><li>Did stakeholders get the information which they wanted to know on priority?</li><li>Did MVQT help me to</li><ul><li>Explore and know what I wanted to know about a feature or a workflow?</li><ul><li>How fast I was here to know and learn this?</li><li>How did I develop my tests incrementally? Did I? If not, then, is it a MVQT?</li></ul></ul><li>Did MVQT help me to know</li><ul><li>Am I aligned and in sync with expectations of my stakeholders and customers who are using the software product I'm testing and automating?</li></ul><li>Did the MVQT help me </li><ul><li>In collecting the critical information in a given context for the scope of testing and automation?</li><li>Do the learning and outcome from this MVQT help to reinforce the validated learning of customer?</li></ul><li>Do MVQT result support the outcome of Unit Testing result?</li></ul><div><br />The tests in MVQT has to be consistently revised and evaluated to keep it as a MVQT. Note this,<b><i> not all tests are MVQT</i></b>. If the number of MVQT is growing to a part of feature or to a feature, it is time to think about what is MVQT for you.</div><div><br /></div><div>The "minimum" tests are highly effective and it helps me learn and test better technically and socially.</div><div><br /></div><div><br /><br /></div><h3 style="text-align: left;">MVQT and Testing</h3><div><ul style="text-align: left;"><li>Sanity or Smoke Tests</li><ul><li>The set of MVQT which helps me learn can the build be taken further testing</li></ul><li>MVFT - Minimum Viable Questioning Functional Tests</li><ul><li>Apply this to a feature or a workflow or to that part which can be evaluated with minimum tests for its functionality</li><ul><li>To update is this aligning to the validated learning of customer [stakeholders]</li></ul></ul><li>MVPT - Minimum Viable Questioning Performance Tests</li><li>MVUT - Minimum Viable Questioning Usability Tests</li><li>MVAT - Minimum Viable Questioning Accessibility Tests</li><li>MVTxT - Minimum Viable Questioning Tester's Experience Tests</li><li>MVST - Minimum Viable Questioning Security Tests</li><li>MVAF - Minimum Viable Questioning Automation to a Feature</li><li>MVLT - Minimum Viable Questioning Localization Tests</li><li>MVUIT - Minimum Viable Questioning UI Tests</li></ul><br />You add more of this to your list and context.</div><div><br /></div><div>In a way, MVQT should ask and look for the testability, automatability and observability. If this is not happening, then there is no possibility of saying I have got my MVQT.</div><div><br /></div><div>More importantly, in the CI-CD ecosystem, MVQT pays a major role. <b><u><i>If I should have my tests in the CI-CD pipeline, then, the MVQT is the way and it focuses on a targeted area to evaluate it</i></u></b>. Else, it is hard, impractical and not possible to test in CI-CD eco system by delivering continuously.</div><p></p><p><br /></p><h3 style="text-align: left;">Ask and Review for MVQT</h3><p>Ask for MVQT, when you review these:</p><p></p><ul style="text-align: left;"><li>test strategy, test framing, test design, test ideas, test cases, test plan, test architecture, test engineering, testing center of excellence, and test code</li></ul><p></p><p>For example,</p><p></p><ul style="text-align: left;"><li><i>What is the minimum viable questioning performance tests that you have got to test this feature?</i></li><li><i>What is the minimum viable questioning performance tests that you have got to test this workflow?</i></li><li><i>What is the minimum viable questioning security tests that you have got to test this feature?</i></li><li><i>What is the minimum viable questioning GUI tests that you have got to test this feature?</i></li><li><i>What is the minimum viable questioning contract tests that you have got to test this end point?</i></li></ul><div>Likewise, <i><b>What is the minimum viable questioning automation tests that you have got to test this feature?</b></i></div><div><br /></div><div>Ask how these tests qualify as MVQT in this context of testing and automation?</div><p></p><p>This should help you to see how effective is the test strategy in a given context.</p><p>Importantly, <b><i>the MVQT and its effectiveness is a testability to test your tests</i></b>.</p><h3 style="text-align: left;"><br /><br /></h3><h3 style="text-align: left;">The Credit is to Me</h3><p>I'm not sure if the idea what I'm saying here in this blog post is practiced by other test engineers. I have not seen this being discussed about it in public forum. I have not come across it in my awareness and to the exposure I have put myself.</p><p>Hence, I will take this credit to me. Giving the credit honestly is not a common sight and practice. I have not got my due credits for using the ideas, thoughts and work that I have come up with.</p><p>So, I make it as a open letter and call out that credit for this idea, thought, concept, and practice will be to me when you listen, use and practice it.</p><p><br /></p><p><br /></p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-7028399228927965110.post-6007816192739619672023-11-11T23:17:00.002+05:302023-11-12T19:33:01.776+05:30Who is a Developer?<p> </p><h3>Who is a Developer? </h3><p>I understand the developer is the one who builds.</p><p></p><ul><li>A product owner builds the product by bridging the gap between market, business and consumer.</li><li>A programmers builds the product by programming.</li><li>A tester builds the product by testing it with her/his tests and programming the test code.</li><li>A devops engineer builds the product by building and managing the pipelines and environment.</li><li>A DBA builds the product by writing and managing the database environments.\</li><li>A support engineer or executive builds the product by troubleshooting and listening to the problems being reported.</li></ul><div><br /></div>Who else do you see in developing the software system and its product as a solution?<div><br /><div>If you see,</div><div><ul style="text-align: left;"><li>With the skills and expertise we have and build consistently,</li><ul><li>Each of us will work with our expertise in a particular space of the problem solving as a team and business.</li></ul></ul><br />Each of us develop some kind of artefact that are combined to build and ship a usable software. To me we all are developers with a specific roles. As a developer we upskill consistently and catchup with current and upcoming changes.</div><div><b><br /></b></div><div><b><br /></b></div><div><b>Note</b>: The above said are not the only tasks of these roles. Each do beyond this in developing & continuous delivery of a usable software.</div></div><div><br /></div><div><br /></div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-7028399228927965110.post-56881233106241562192023-11-04T17:24:00.003+05:302023-11-19T21:11:14.134+05:30The Agile & Testing: We're Pulled & Hidden from "Shift Right"<p><br /></p><h3 style="text-align: left;">Do I Understand the Agile?</h3><p style="text-align: justify;">If I work in a project which follows the Waterfall approach, I still can apply the Agile Manifesto and the 12 principles behind it.</p><p style="text-align: justify;">But, am I adaptable to the change and uncertainty for better ways of developing software and helps others do it? This shows the mindset of me and others involved. Have you heard this line, "<i>every project that claims Agile, it has [a small] waterfall in it?</i>".</p><p style="text-align: justify;"><u>Agile Software Development is not wrong!</u> It has values in what it says and for what it stands. I have no doubts in it! It describes how I perform a activity and respond to a situation in the software development.</p><h4 style="text-align: left;"><br />Frameworks and Methodology</h4><p></p><p style="text-align: justify;">We have picked the frameworks which are under the Agile's umbrella. We pick the methodology that suits our work and have customized it to our contexts.</p><p style="text-align: justify;">This is what Agile also say, <i><u>the team will always need to adapt its use of a framework to fit properly in its context</u></i>. </p><blockquote><p style="text-align: justify;">Dr. Alistair Cockburn, says, <i>the Agile methodologies are the conventions that a team choses to follow in a way that follows Agile values and principles</i>.</p></blockquote><h4 style="text-align: justify;"><br />The Agile is Not a Problem!</h4><p style="text-align: justify;">We do not collaborate well in solving the problem and say Agile is not working and creating more problems.</p><p style="text-align: justify;">In reality, how we respond to the uncertainties and what got prioritized, it will derail us. We derail from what we should be achieving as team [and as an individual] in a given time frame and context.</p><h4 style="text-align: justify;"><span style="text-align: left;"><br />The Agile as I Understand</span></h4><p>In a nutshell, the Agile is about</p><p></p><ul style="text-align: left;"><li>Balancing effectively with collaboration in the context we are.</li><li>Identifying, learning and solving the problem as a team no matter where we are in the project's timeline.</li><ul><li>Being ready to face the uncertainty and responding to the changes.</li><li>See the progress and value in what we do as a team and bring the "continuous" in practice.</li><li>Having the consistency in the excellence of our work and the value we deliver with it.</li></ul></ul><div style="text-align: justify;">Read the Agile Manifesto and its 12 principles. <u>It is the day-to-day life of a team who works to build a software in a continuous delivery having the values to a stakeholder</u>.</div><h3 style="text-align: left;"><br /></h3><div><br /></div><div><br /></div><h3 style="text-align: left;">Testing and the Agile</h3><p style="text-align: justify;">Testing is a collaborative activity.</p><p style="text-align: justify;">The Agile also talks about collaboration as an essence and necessity in the continuous delivery of the software we develop. </p><p style="text-align: justify;">The Agile talks about short interval consistent deliveries.</p><p style="text-align: justify;"></p><ul><li>How should I fit myself as a Test Engineer here?</li><li>How should I upskill myself to deliver in short intervals as part of continuous delivery? </li></ul><p></p><p>I ask and say to my test teams,</p><p></p><ul style="text-align: left;"><li>Think of delivering value consistently in the shorter intervals.</li><ul><li>Rather saying -- <i>releases in short intervals</i>, call it out as <i><u><b>a value to a stakeholder in regular short intervals as a continuous delivery</b></u></i></li><ul><li>It sounds positive, affirmative and profound.</li><li>Upskill and help your peers to build the skills. List the skills, let us discuss and start our practice on it.</li></ul><li>This mindset changes the perception in how you approach the testing.</li></ul><li>In these short intervals,</li><ul><li>Let us prevent bugs in the ways we can think of.</li><ul><li>Finding bugs also need resources.</li><li>Let's start investing in building and improvising the resources to prevent bugs.</li><ul><li>How can we prevent and where to begin?</li></ul></ul></ul><li>For what should we test and automate other than functionality in these short intervals?</li><ul><li>What is the priority?</li><li>If I do not test and automate for evaluating any particular quality criteria, does it impact the value to a customer in what we deliver?</li><li>If I do not test and automate on a particular seam and system's boundary interface, does it impact the value to a customer in what we deliver?</li></ul><li>How am I collaborating with stakeholders?</li><ul><li>Who are my stakeholders?</li></ul></ul><p></p><p><br />In short, I see the Software Test Engineering practice in the Agile setup as this,</p><p></p><ul style="text-align: left;"><li>We [team] exploring technically and analytically;</li><li>Evaluating collaboratively in the software development and engineering, by having the effective staged feedback loops in place to improvise the delivery;</li><li>By responding actively to the changes and uncertainties that comes anytime in the timeline of a delivery and practice;</li><li>Thus, the consistent continuous delivery is delivered with value to the stakeholders.</li></ul><p></p><p><u><b><br />Note</b></u>: <i>This is my observations so far having worked with 46+ startups in my software test engineering practice</i>.</p><p><br /></p><h3 style="text-align: left;">We're Pulled and Hidden From - Shift Right</h3><p>I and you are said to Shift Left and start the testing from the inception of a feature's discussion. And, further we're said, <i>we have to shift left from right</i>.</p><p>Did the Agile manifesto and its 12 principles say explicitly to Shift Left from Right? Who said that?</p><p><i><b>What I see, How </b><b>I see and understand the Right and Left here, matters a lot!</b></i> If I see the Right within the delivery loop, I'm setting a trap to myself to be late and lag back. And, probably this trap gets set in the work, that is, to see the Right in the [continuous] delivery loop. At certain context in continuous delivery loop, starting from Right is a need and necessity.</p><p><b><u>I see the Right outside the [continuous] delivery loop, as well</u></b>. In this Right, I see:</p><p></p><ul style="text-align: left;"><li>What is coming up in the tech stack and technology?</li><ul><li>How this is a need?</li><li>How this [is to be and] will be adapted by tech solutions we deliver?</li><li>How should I upskill myself here so that I fit into the loop of continuous delivery?</li></ul><li>In short, <b><u><i>the Right should be observing the trend and its analysis so that I'm aligned being agile</i></u></b>.</li><ul><li>I will be skilled to test and automate in this change and uncertainties.</li></ul></ul><div>The Agile never said to completely move or step out of the Right. Or, I have not read it in its manifesto and principles. </div><div><br /></div><div>You see and learn, what is your Right and ask am I aligning to it for being agile in the Agile. After all, the Agile says adapting to the changes and uncertainties, yet delivering the value to the stakeholders consistently in short intervals.</div><div><br /></div><div>If seen closely, the Agile is also about balancing, isn't it? <b><i>If I'm moving towards Left and me standing in the Left, am I balanced?</i></b></div><div><br /></div><div>In short, <b>we are being pulled and hidden from Shift Right</b>.</div><div><br /></div><div><u>Shifting Right is important to see what is outside the delivery loop</u>. <b><i>Are you seeing your Right? If yes, what is the trend analysis you see? Where you need to upskill?</i></b></div><div><br /></div><div><b><i>Shift Right can also be to look what you can bring into your Shift Left for better development and delivery</i></b>. For example, what you see in the market and business space for your product? Do I have anything to add now in Left from this learning? Do you see that? Are not we missing this when we pulled and hidden from Shift Right?</div><div><br /></div><div>Left or Right, if I do not respond, adapt and deliver for the changes, I need to ask, -- <b><i>Am I agile and testing in the Agile?</i></b></div><div><i><br /></i></div><div><i><br /></i></div><div><i><br /></i></div><div>To conclude, know what is your Shift Right while you are delivering values in your Shift Left. Strike the balance and ace.</div><div><br /></div><div><br /></div><div><br /></div><p></p>Unknownnoreply@blogger.com2tag:blogger.com,1999:blog-7028399228927965110.post-26332174324052927442023-11-04T17:03:00.002+05:302023-11-04T17:44:57.555+05:30Exploring to know the Web API - 101<p> </p><p>Here is what I look when I'm exploring to learn an end point of <b>web API</b> for first. I'm talking of the Web API that uses HTTP protocol.</p><p></p><ol style="text-align: left;"><li>What is the purpose of this end point?</li><ul><li>How it helps and to whom?</li></ul><li>Who uses this end point?</li><li>Identify the host of the end point</li><li>Identify the path of the end point</li><li>Identify the API's end point</li><li>Identify the version of the end point</li><ul><li>Know the different version that are available for an end point</li><ul><li>What are active?</li><li>What are inactive?</li><li>Who are the consumers using these different versions of this end point?</li><li>What's the difference between these versions?</li></ul></ul><li>Identify the HTTP method of the end point</li><ul><li>Know the different HTTP methods this end point supports</li></ul><li>Identify the resources the end point interact with for CRUD activity</li><li>Know the HTTP Request headers it needs, uses and good to have</li><li>Know the HTTP Request payload or data and its format</li><li>Know the HTTP Response headers and good to have</li><li>Know the HTTP Response Status Code and what actually it should be returning</li><li>Know the HTTP Response content and format returned</li><li>Does this end point enforces any contract with the consumer?</li><li>Know the intent of your test on this end point</li><ul><li>One test in one request is useful unless the context demands otherwise</li></ul><li>Is there anything cacheable in the end point's response?</li><li>How the request payload is?</li><li>How the response payload is?</li><li>What is the language in which the end point is written?</li><li>What is the language in which the resources that end point updates is written?</li><li>Any framework used to build this end point?</li></ol><div><br /></div><div>On learning this, I continue to next phase in exploring and knowing the web API.</div><div><br /></div><div><br /></div><div><br /></div><p></p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-7028399228927965110.post-87956529879845387952023-10-27T01:23:00.001+05:302024-02-05T10:48:25.681+05:30The Actual Performance Bottleneck is a Test Engineer's Awareness & Practice<p> </p><p style="text-align: justify;">The bottlenecks are necessity. We cross bottlenecks everyday; just, we do not observe it. If you have not read my today's thought on the bottleneck, read it <a href="https://testinggarage.blogspot.com/2023/10/understanding-the-bottleneck-in-software-engineering-and-testing-ravisuriya.html" target="_blank">here</a>. </p><div><span style="text-align: justify;">The seventh question from season two of 100 Days of Skilled Testing is:</span></div><p></p><blockquote><p style="text-align: justify;"><i>What are the common performance bottlenecks, and how do you go about pinpointing and resolving them?</i></p><p></p></blockquote><p><br />I have a different opinion and thoughts to share on this question. It may look weird, but, I see this is the reality.</p><p style="text-align: justify;">Let me share what happens to a practicing Test Engineer when going through a bottleneck in practicing a subject to upskill.</p><ul style="text-align: left;"><li>The time taken to accomplish tasks and its milestones looks unusually high - the other extreme end.</li><ul><li>Some attempts will not even kick off on entering the bottleneck for several reasons. </li><ul><li>One of the reason is the fear and what my peers think if I fail to accomplish it.</li><ul><li>I lose the focus.</li></ul></ul></ul><li>It leads to lose of interest, unhealthy confidence and the inconsistency.</li><ul><li>The procrastination kicks in.</li><li>Without no self-motivation, determination and consistency, we give up; clog and remain in the bottleneck. One will fail to make use of the bottleneck to scale self.</li><ul><li>One do not go back or come out of it.</li><ul><li>We start to blame the bottlenecks.</li><li>But, we fail to understand that the bottlenecks are part of the system.</li></ul></ul></ul></ul><div><p style="text-align: justify;"><br />What makes one to have a hard time with available bottleneck is,</p><p style="text-align: justify;"></p><ul></ul></div><ul style="text-align: left;"><li>Ignoring the bottleneck.</li><li>Not being aware of the benefits and impact from the bottleneck's existence.</li><ul><li>Its effects are adverse in a longer run</li><li>Likewise, its benefits are immense in a longer run</li></ul></ul><div><p style="text-align: justify;"><span style="text-align: left;">If seen, the existence of bottleneck has two extreme ends -- good and not so good. </span><span>The bottlenecks cannot be eliminated or eradicated; it can only be managed with awareness and knowing how to adapt and scale through it.</span></p></div><p></p><p style="text-align: justify;"><br /></p><p><br /></p><h3 style="text-align: left;">Me, The Actual Performance Bottleneck</h3><p>This may appear as a critical self evaluation. But, it is not. In my experience and practice, I'm learning consistently,</p><p></p><ul style="text-align: left;"><li>If there is any difficulty in communicating about the tests and identifying the information from my tests,</li><ul><li>It is do with me for first.</li><ul><li>I will have to communicate it and why it is so.</li></ul></ul><li>If I'm not exposing myself to a subject's area and its practice by putting the subject to test,</li><ul><li>I'm the bottleneck.</li></ul><li>Further, the other bottlenecks that I encounter from all other systems, it easily compounds the problem and its impact.</li></ul><div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><b><i>Did you see the bottleneck is a necessity?</i></b> It drives me to scale and be operable. If no bottleneck, may be, I will not expose myself to the awareness to identify and learn the information.</div></div><p></p><p style="text-align: justify;">Call out any testing with a name, for first I will be the bottleneck to myself and to my testing, <b><i>if I'm not aware of it and not practicing in it</i></b>.</p><p style="text-align: justify;">For example, I'm asked and taught to do functional testing at a GUI layer. Why I'm not asked and taught to do the performance and security tests at a GUI layer? Or, why I do not think of it? Should someone say me to think of it, practice it, and, do testing for the same? If the floor and industry do not push me to do so, I will have to open myself to create that opportunity and awareness. <b><i>Did you see that, this is my performance bottleneck?</i></b></p><p style="text-align: justify;">To test, evaluate and identify [pinpoint] the performance bottleneck in the software system, I will have to practice Performance Engineering & Testing. Only then, I will be able to identify its existence precisely before pinpointing. If I do not practice by building and refining the awareness, I cannot explain what I'm pinpointing.</p><p style="text-align: justify;">I pinpoint myself [the bottleneck] first, if I identify wisely what is the performance bottleneck. If I fix my bottlenecks, I will be able to identify bottlenecks in the system that I test.</p><p style="text-align: justify;">Let me practice the performance engineering and help my team, and community to practice it. If not, I will not be in position to name a bottleneck, forget pinpoint it. I will just follow, what others say and think it is right.</p><p style="text-align: justify;"><br /></p><p style="text-align: justify;">Note: <i>I share the above from my personal experience in testing practice and also from reading [& answering] the performance testing questions that are posted in the communities social spaces. I see fixing this and practicing better here is important for first, than identifying and solving the bottlenecks in software system.</i></p><p style="text-align: justify;"><br /></p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-7028399228927965110.post-34834725427565430442023-10-21T20:11:00.002+05:302024-02-05T10:48:44.470+05:30The "Bottleneck" in a Test Engineer's Eyes<p style="text-align: justify;"> </p><h3 style="text-align: justify;">Preference to Bottle Over Jar! Why?</h3><p></p><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Have you heard Jar Neck anytime when describing a problem or solution?</div><p></p><p></p><ul style="text-align: left;"><li style="text-align: justify;">I have heard Bottleneck often and consistently; but, not Jar Neck . Why? </li><li style="text-align: justify;">Be it in Software Engineering or day-to-day life problem solving description,</li><ul><li style="text-align: justify;">The Bottleneck is referenced and not a Jar Neck.</li></ul></ul><p></p><p style="text-align: justify;">Looks like people want Bottle but not the Bottleneck speed and benefits. Bottle without its neck is a jar?!</p><p></p><div style="text-align: justify;"><span style="font-weight: 700;"><br /></span></div><b><div style="text-align: justify;"><b><br />Bottleneck exist for better controllability</b>.</div></b><p></p><p></p><ul style="text-align: left;"><li style="text-align: left;">In a bottle, the bottleneck is a solution! It is not a problem!</li><ul><li style="text-align: left;">It is to mitigate any risk and problem that arises from the flow of content in the bottle.</li></ul></ul><p></p><p style="text-align: justify;">Yet we describe, learn and communicate the neck of a bottle as a relativity and analogy to a problem. </p><p></p><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Are you aware of Gateway in the software system?</div><p></p><p></p><ul style="text-align: left;"><li style="text-align: left;">The Gateway can be seen as a neck of a bottle which controls the incoming requests and outgoing response.</li><li style="text-align: left;">Gateway is a necessity.</li><ul><li style="text-align: left;">We need Gateway to be adaptable in size of its neck based on traffic volume it is handling. Here, the gateway's neck size should adapt and scale contextually.</li><ul><li style="text-align: left;"><i><b>When describing a problem, we are talking about how this bottleneck size <u>which is not adaptable for the context</u></b></i>.</li><li style="text-align: left;">That adaptability has to be built in engineering to scale in any dimensions and magnitude.</li><ul><li style="text-align: left;">When this is not done, we equate the software system's problem to a bottleneck as a analogy, <u>which is incorrect</u>! The bottle has got its size and its neck size fixed <u>for a purpose and as a solution</u>.</li><ul><li style="text-align: left;">The context of a bottle and today's any systems are different.</li><ul><li style="text-align: left;">It is good to draw similarities from General Systems Thinking and observations.</li><li style="text-align: left;">But the solution cannot be generic to all systems; it has to be contextual. The software system has to have its contextual solution.</li></ul></ul></ul></ul></ul></ul><p></p><p style="text-align: justify;"><br />So, next time when someone in your team or network talk about bottleneck, do share them bottleneck is for better controllability. <b><i>Having a contextually resizable and adaptable bottleneck is the need for Software Engineering; not the elimination of bottleneck</i></b>.</p><p style="text-align: justify;">In fact, a software system should have and will have a bottleneck in a point. And, this bottleneck will be adaptable to the context for having what it should let through and process.</p><p style="text-align: justify;">Is the runway of an airport a bottleneck when it is compared to a sky? Is that a solution or a problem? Likewise, the ship will have a defined route path and it does not sail without a route path. Is this a bottleneck to ship and its business? A elevator can accommodate the defined number of people or kilograms allowed, and not beyond that to move. Is that a bottleneck? The esophagus in human body has a size which medical science observes as normal and acceptable; any deviation from that size measurement, the medical science test investigates it as a risk and problem. Why? Is the circumference size and length of esophagus a bottleneck to human anatomy and physiology system?</p><p style="text-align: justify;"><u>The engineering solution will and should have a bottleneck at a point</u>. Having a adaptable bottleneck to the context is one what tries to accomplish in a software system's scalability and operability.</p><p style="text-align: justify;"><br /></p><p style="text-align: justify;">Please, do not equate solving a "bottleneck" situation with Agile practice. Does it look like a joke? I will not be surprised if someone says bottleneck problem is solved if practiced Agile.</p><p style="text-align: justify;"><br /></p>Unknownnoreply@blogger.com2tag:blogger.com,1999:blog-7028399228927965110.post-54511391171505773282023-10-21T01:38:00.002+05:302024-02-05T10:49:06.550+05:30The Internal Metrics of High Performance Mobile App<p> </p><p>Do you have a question -- <i>What are KPIs and Metrics and the differences between them? </i>Then read <a href="https://testinggarage.blogspot.com/2023/10/what-are-kpis-and-metrics-engineering-testing-ravisuriya.html" target="_blank">this</a> blog post. It will help in knowing the differences and how each help the other mutually and remain exclusive.</p><p>The sixth question from season two of 100 Days of Skilled Testing is:</p><p></p><blockquote>How do you determine the essential performance metrics and Key Performance Indicators (KPIs) when assessing the performance of a mobile native app?</blockquote><p></p><p><br /></p><h3 style="text-align: left;">Why I Focus on Metrics Here?</h3><p>Here, in this blog post, I will focus on Metrics and not KPIs. If you have read the above referenced blog post on KPIs and Metrics, you will learn,</p><p></p><ul style="text-align: left;"><li>The KPIs are defined objectives by the business.</li><ul><li>Knowing this is important.</li><li>But, to accomplish the KPIs objective, one has to extract the contextually suitable metric from the system and have to evaluate it.</li></ul></ul><p></p><p>As a test engineer, I evaluate from technical perspective with an orientation of business thoughts.</p><p></p><ul style="text-align: left;"><li>So, the identifying and defining the metrics makes more sense to my role.</li><ul><li>Hence, I pick the Metrics.</li></ul></ul><p></p><p>A KPI example for a mobile app,</p><p></p><ul style="text-align: left;"><li>The set percentage [what percentage?] of five start rating in store with positive emotions in a review.</li></ul><p></p><p>What may be the KPIs, I will have to correlate it with [a] metrics so that we work towards accomplishing the KPIs set.</p><div><br /></div><h3 style="text-align: left;"><br />What is Performance to Mobile Native App?</h3><p></p><h4 style="text-align: left;"><br />The Native App</h4><p></p><p>A native mobile app is an app developed for a particular mobile device or platform like Android and iOS. The native apps developed for one platform cannot be run on another platform. That is, Android app cannot run on iPhone, and vice versa.</p><p>The Android apps are written in Kotlin, today. The iOS apps are written in Swift. The native apps can be developed in a way that it can run both in offline and online mode based on its business objectives.</p><p>An example of the native app which we can easily understand is,</p><p></p><ul style="text-align: left;"><li>WhatsApp for Android and iOS devices</li></ul><p></p><p><br /></p><h4 style="text-align: left;">Performance and Native App</h4><p>For first, my mobile device is not my customers device. The Android device fragmentation makes it much more challenge with computing power offered for devices by OEMs.</p><p>Though, iOS have its fragmentation on OS version and device's computing power, it is not huge when compared with fragmentation of Android devices. This is a challenge for Android mobile app in all aspects and especially in performance.</p><p>The performance will indicate different advantage, risks and problems to a native app on a platform and its devices. The performance can be classified into different areas for native mobile apps.</p><p>It can range from, being deep technical areas to the experience a user expresses in using the app in a given context. <b><i>Everything is performance here!</i></b></p><p>Hence it is not easy when talking about performance in the mobile's app space. While it is so ambiguous and hard subject, how one can pull the metrics for the performance here?</p><p>From a technical perspective, the word "performance" is not specific; it is vague. If one says, the mobile app is performing well, what does it mean?</p><p></p><ul style="text-align: left;"><li>Is it fast?</li><li>Is it consuming low battery power?</li><li>Is it consuming less memory?</li><li>Is it not consuming much network?</li><li>Is it smooth to interact, responsive, and no jank experience?</li><li>Is it intuitive and secure?</li><li>Is it number of crashes?</li><li>Is it having a consistent update and bug fixes?</li></ul><br />What you say?<p></p><p></p><div>All these leads to a question - <i><b>What is a high performance app?</b></i> You ask this question to yourself and to your tech team, business team and consumers. Figure out what is a high performance app to them. This helps!</div><div><br /></div><div><br /></div><h4 style="text-align: left;">The Common Metrics - Android and iOS</h4><div><br /></div><div>Below are some of the common metrics for which a Test Engineer extracts data and evaluate it via testing and automation.</div><div><br /></div><div><ol style="text-align: left;"><li>App Install Time</li><li>App Launch Time</li><ol><li>Cold Start</li><li>Warm Start</li><li>Hot Start</li></ol><li>Private Memory Size of App on available Heap</li><li>Number of Views</li><li>Garbage Collection - Frequency</li><li>Data Residual Size and Sharing</li><li>Network Payload Size</li><li>Energy Consumption in Workflows</li><li>Wakelocks and its Impact</li><li>Frames Skipped</li><li>Open Threads and Processes</li><li>Storage & Data Size</li><li>App Size</li></ol><div><br />And, more. Each of these are own deep area for analysis and tuning the performance.</div><div><br /></div><div>I have been practicing this for last 11 years in my testing. It is one of my research areas in Software Testing & Engineering</div><div><br /></div><div>These days, instrumentation also offers different metrics for the mobile apps which can correlate with the KPIs set for the native mobile apps.</div></div><div><br /></div><div><br /></div><div><br /></div><div>To conclude, start understanding the internals of Android and iOS. It opens up you to the performance and practice of high performance apps.</div><div><br /></div><div>I'm happy to share if you are interested in practicing these subjects and testing. Let us connect and converse!</div><div><br /></div><div><br /></div><p></p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-7028399228927965110.post-13470158838737338812023-10-18T23:12:00.004+05:302024-02-05T10:49:27.018+05:30What are KPIs and Metrics?<p style="text-align: justify;"> </p><p style="text-align: justify;">I use to have this question - <i><b>Are KPIs and Metrics the same?</b></i> Especially when I started to learn and practice the Performance Testing, this question bounced back to me often.</p><p style="text-align: justify;">Do you have this question?</p><p style="text-align: justify;"><br /></p><h3 style="text-align: justify;">The Use of KPIs and Metrics</h3><p style="text-align: justify;">When business and stakeholders talk about it so much, there should be a value out of it. What is that value? Why it is important to identify and capture the KPIs and Metrics?</p><p style="text-align: justify;">The KPIs and Metrics are derived from data we collect. These data are processed to extract and normalize, so that, it is in a state as expected by the consumer for making a decision.</p><p style="text-align: justify;">The stakeholders will make decisions and take actions referring to KPIs and Metrics. For example,</p><p></p><ul style="text-align: left;"><li style="text-align: justify;">The number of Daily Active User (DAU) is a KPI and also a metric.</li><ul><li style="text-align: justify;">But, a metric cannot be a KPI</li></ul><li style="text-align: justify;">How many of this DAU, closed the transaction within five seconds using a wallet?</li><ul><li style="text-align: justify;">It is a metric; not a KPI.</li></ul></ul><div style="text-align: justify;"><br />Another example,</div><div><ul style="text-align: left;"><li style="text-align: justify;">KPI</li><ul><li style="text-align: justify;">How many users installed the latest version of the mobile app and have signed in?</li><ul><li style="text-align: justify;">If there no active users on latest version that indicates a kind of risk and problems to the business.</li></ul><li style="text-align: justify;">Reopened Tickets in Customer Care</li><ul><li style="text-align: justify;">This indicates there is something going wrong</li></ul></ul><li style="text-align: justify;">Metric</li><ul><li style="text-align: justify;">What is the average time taken to see a streaming screen for users in 4G data network?</li><ul><li style="text-align: justify;">If this is not captured, there is no data for business to establish a relationship with KPI set.</li></ul><li style="text-align: justify;">Average Reopened Tickets in a customer service</li><ul><li style="text-align: justify;">The distribution and time towards lower number</li><li style="text-align: justify;">The distribution and time towards higher number</li></ul></ul></ul><div style="text-align: justify;"><br />KPIs and Metrics are not the same while both have quantitative measurement. Both are different. Identifying and knowing the difference between them in your context is important.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">They go hand-in-hand when setting a direction and action. So that, the business and stakeholders realign to the goals and objectives defined.</div></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br /></div><h3 style="text-align: justify;">KPIs vs Metrics</h3><div style="text-align: justify;"><br /></div><div><div class="separator" style="clear: both; text-align: center;"><div class="separator" style="clear: both; text-align: left;"><div class="separator" style="clear: both; text-align: left;"><div class="separator" style="clear: both; text-align: left;"><div class="separator" style="clear: both; text-align: left;"><div class="separator" style="clear: both; text-align: left;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgyUU3TxllfNsFdmRuQpP8tUnfiieARzivyPcu5UcSTuNna9bUWHFYe8ADJs9pRVHG-u0rzhxRjs6yQQESxgB8YaPrnhTug2I4AbVsqKom60BD3j2EsfjnrR3gW37k59s8Gq9fmTkhit7vj2QZdee2t6w_9UWeZlE-FewLSST76MCrdw8ozftXRTtygj3I" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="627" data-original-width="778" height="515" src="https://blogger.googleusercontent.com/img/a/AVvXsEgyUU3TxllfNsFdmRuQpP8tUnfiieARzivyPcu5UcSTuNna9bUWHFYe8ADJs9pRVHG-u0rzhxRjs6yQQESxgB8YaPrnhTug2I4AbVsqKom60BD3j2EsfjnrR3gW37k59s8Gq9fmTkhit7vj2QZdee2t6w_9UWeZlE-FewLSST76MCrdw8ozftXRTtygj3I=w640-h515" width="640" /></a></div><br /><br /></div></div></div></div><br /><br /></div><div style="text-align: justify;">To conclude this post, investigate your metrics and question why the chosen KPIs. It will help you to design your Test Models and identify the tests in given context.</div></div><p></p><div class="separator" style="clear: both; text-align: center;"><div style="text-align: justify;"><br /></div></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br /></div><p></p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-7028399228927965110.post-45252694924992702102023-10-17T22:44:00.002+05:302024-02-05T10:49:42.598+05:30Software Engineering - The Unquestioned Understanding of Client in Testing<p style="text-align: justify;"><br /></p><h3 style="text-align: justify;">Client - The Unquestioned Understanding</h3><p style="text-align: justify;">When asked or said "client" or "client-side", most of us assume or take it as:</p><p></p><ul style="text-align: left;"><li style="text-align: justify;">Web page displayed in the browser</li><li style="text-align: justify;">Mobile apps - Android and iOS apps</li><li style="text-align: justify;">Desktop applications</li></ul><div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">I see <b><i>this is one of the unquestioned understanding and assumption in the Software Engineering</i></b>. While it is not wrong, the client does not always mean -- a web page displayed on browser, mobile apps, desktop applications, a terminal, etc.</div></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">The client is one of the subjects which we have not attempted enough to understand in Software Testing & Engineering.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">A client is one who consumes the service in form of a response, and then does what it has to do.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br /></div><h3 style="text-align: left;"><div style="text-align: justify;"><span style="font-size: 18.72px;"><br /></span></div><div style="text-align: justify;">Client - The Contextual Entity</div></h3><div style="text-align: justify;"><br /></div><div style="text-align: justify;">The entity which takes the a client place [role] is entirely based on the context. That web page displayed on a browser is a client per some model. So is the mobile apps. A service looking for data from Redis is a client too.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">The client can be within the backend system which requests another entity to process its request and awaits for the response. This client is not always a mobile app, web page on a browser, or a terminal where I'm working with commands. Do you see that?</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Next time when you hear the word client, ask for the context and know who is the client that is being discussed.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br /></div><h3 style="text-align: left;"><div style="text-align: justify;"><span style="font-size: 18.72px;"><br /></span></div><div style="text-align: justify;">Client's Awareness in Performance Tests</div></h3><div style="text-align: justify;"><br /></div><div style="text-align: justify;">By now, you should be breaking your assumption and the myths around the word "client".</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">In testing for Performance, it is critical to be aware who is client, when and how? Evaluating this client will not be like evaluating a web page on a browser or the mobile apps.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">The fifth question from season two of 100 Days of Skilled Testing, is:</div><p></p><blockquote><p style="text-align: justify;">How does client-side performance testing contrast with server-side performance testing, particularly in their objectives and area of emphasis?</p></blockquote><p style="text-align: justify;">Hope now you will question when said client-side performance,</p><p></p><ul style="text-align: left;"><li style="text-align: justify;">What client are we talking here?</li><li style="text-align: justify;">Where do this client sit in the system's architecture.</li></ul><p style="text-align: justify;">Based on this information,</p><ul style="text-align: left;"><li style="text-align: justify;">How the tests for performance is approached and executed for a client, differs.</li><li style="text-align: justify;">What is collected and observed to evaluate the performance of a client, differs.</li></ul><p></p><p style="text-align: justify;"><br /></p><p></p><div style="text-align: justify;"><br /></div><div style="text-align: justify;">This blog post is not to illustrate the different clients and how evaluate it for performance. When the question talks about a particular client and in a context, I will share the approach, and how to evaluate the same.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">To end, have we explored the clientless interface? How to test this interface?</div><p></p><p style="text-align: justify;"> </p>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-7028399228927965110.post-38498049469785901712023-10-16T23:37:00.009+05:302024-02-05T10:50:06.092+05:30Performance & Tests: Getting Started and Data Analysis<p style="text-align: justify;"> </p><p style="text-align: justify;">On running tests,</p><p style="text-align: justify;"></p><ul><li style="text-align: justify;">We will have data (information) as one of the byproduct.</li><li style="text-align: justify;">Analyzing the data of the integrated sub-systems in isolation and correlation,</li><ul><li style="text-align: justify;">It will lead us to a technical analysis on each integrated system.</li></ul></ul><div style="text-align: justify;">In the report, we draft this analysis along with actions to be taken.</div><p></p><p style="text-align: justify;"><b>Note</b>: When said sub-systems do not ignore or skip the client or consumer; the system does not comprise just server.</p><p style="text-align: justify;"><br /></p><h3 style="text-align: justify;">No Golden Rule</h3><p style="text-align: justify;">There is no one way to do a testing. Likewise, there is no one way or the golden rule to test for performance. It is contextual and depends on what I want to learn.</p><p style="text-align: justify;">In fact, in few contexts, we can have a value adding performance test with just one request. Just, I should be well aware of -- <i>what is that I want to know and learn from this test</i>.</p><p style="text-align: justify;">That said, there are multiple interfaces where we can observe, analyze and learn from the performance data collected.</p><p style="text-align: justify;">The fourth question from season two of 100 Days of Skilled Testing, is:</p><blockquote><p style="text-align: justify;"><i>What are your favorite hacks to analyze performance testing results and find anomalies?</i></p></blockquote><p style="text-align: justify;">Well, this question do not mention explicitly if it is for server or client or database or caching or messaging or for what interface of a system. It is a question; but, to me it looks too generic and at a point it looks vague. Having said this, that is how the learning journey and curve starts! </p><p style="text-align: justify;"><br /></p><h3 style="text-align: justify;">Result vs Report</h3><p style="text-align: justify;">What is a result?</p><p style="text-align: justify;"></p><ul><li>Is it an evaluation after a data [information] is put to scrutiny?</li><li>Or, the result is a data that is collected and not yet interpreted?</li></ul><p></p><p style="text-align: justify;">It depends on individual or team and how it being practiced.</p><p style="text-align: justify;"><u>The result is different from a report</u>.</p><p style="text-align: justify;"><br /></p><h3 style="text-align: justify;">Getting Started and Data Analysis</h3><div style="text-align: justify;"><br /></div><div style="text-align: justify;">I should know how the system architecture is designed and orchestrated with its boundaries and interfaces. This helps a lot. What kind of architecture is this? Is it a monolith? If it is monolith, my approach to test for performance differs.</div><div><span style="text-align: justify;"><br /></span></div><div><span style="text-align: justify;">If I'm asked to start the analysis of data for a system that I'm not aware of,</span></div><div><ul style="text-align: left;"><li><span style="text-align: justify;">I will start by analyzing the below indicators on knowing the architecture and the orchestration of the sub-systems for critical business workflows</span></li></ul><ul style="text-align: left;"><ol><li style="text-align: justify;">CPU usage</li><li style="text-align: justify;">RAM usage</li><li style="text-align: justify;">Data I/O</li><li style="text-align: justify;">Network usage</li><li style="text-align: justify;">The Heat and sound dissipated from the hardware which holds and binds</li><ul><li><span style="text-align: justify;">CPU, RAM, Data I/O, Network and tech stacks installed and configured</span></li></ul></ol></ul><div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">It hints me to look further and test investigate, when I observe:</div></div></div><div><ul style="text-align: left;"><li style="text-align: justify;">Having a steady consumption</li><ul><li style="text-align: justify;">What is steady consumption in this context?</li></ul><li style="text-align: justify;">Having a low consumption</li><ul><li style="text-align: justify;">What is low consumption in this context?</li></ul><li style="text-align: justify;">Having a unusual consumption spike and fall of it</li><ul><li style="text-align: justify;">I follow the pattern to study further</li><li style="text-align: justify;">What is considered as knee, spike and fall, in this context?</li></ul><li style="text-align: justify;">Having a zero consumption</li><li style="text-align: justify;">Having a maximum consumption</li><ul><li style="text-align: justify;">What is maximum consumption in this context?</li></ul></ul><div style="text-align: justify;">Having a high consumption doesn't mean a problem. Likewise, having a low consumption does not mean all is well. I have to uncover them to learn what it means in the given context.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">In each of this, there will be a pattern. I will learn them. I will correlate with other sub-systems and learn what they were doing in the said timeline.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Do you recollect this line -- "the architecture should provide the Testability"?</div><div style="text-align: justify;"><ul><li>I <a href="https://testinggarage.blogspot.com/2023/10/what-is-software-architecture-how-it-enables-testability-and-testing-ravisuriya.html" target="_blank">wrote</a> about it in one of the blog posts of Performance Engineering.</li></ul></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">I refer to the below by traversing with the timeline,</div><div style="text-align: justify;"><ul><li>The logs by asking for it</li><li>Data recorded</li><li>Any APMs that are in place</li></ul>I correlate all these with above said indicators.</div></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">This gives me a start. It is one of easiest start that I can have to get started with analysis.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Well this is to analyze at the server end. What about the client [consumer] end? It is simpler and will share in the coming blog posts.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Do you want to know more on this and other strategies that can be used contextually? Let us get connected and converse. I'm happy to share and learn on listening to you. It is fun and awareness!</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br /></div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-7028399228927965110.post-3854272740466762402023-10-11T01:10:00.001+05:302024-02-05T10:50:32.066+05:30Prioritizing Performance & Its Requirements - The Two Engineering Tasks<p> </p><p style="text-align: justify;"><i>How do I gather and prioritize the performance requirements of a student from schools, colleges, universities and society?</i> </p><p style="text-align: justify;">Note that, I said performance. What do performance mean in schools, colleges, universities and society?</p><p></p><ul><li style="text-align: justify;">Any time, you asked this question to self?</li><li style="text-align: justify;">If you are living with children, did this question cross your mind, no matter in what class the children are studying?</li></ul><div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">This is not the question which can be ignored. Also, this question is not precise and to the context. It is the question that is resonated but has no acceptable rational base for whatever context from which it arises. <u>The same when it comes to performance requirements of a software system</u>.</div></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">If you observe closely, the system in which we live, it pushes towards performance for what it thinks as a performance. Isn't it?</div><div style="text-align: justify;"><br /></div><div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">The third question in season two of 100 Days of Skilled Testing is:</div></div><p></p><blockquote><p style="text-align: justify;"><i>How do you gather and prioritize performance requirements from stakeholders and project documentation?</i></p><p></p></blockquote><p><br /><br /></p><h3>Prioritizing Performance & Its Requirements</h3><p><br /></p><div style="text-align: justify;">If you read attentively, the title of this section says - prioritizing performance and its requirements. <b>I did not say</b>, prioritize performance requirements.</div><p></p><p style="text-align: justify;">That said, <b>what is performance for Netflix is not same for the Aadhar system</b>. But, both have prioritized the performance and looks to be aggressive in knowing the requirements for same. Don't you think so?</p><p style="text-align: justify;">We're in the timeline of Shift Left.<br /></p><ul><li>How to shift with performance to left? </li><li>What all in the system should focus on the performance in the Shift Left?</li></ul><div><br /></div><h4><br />MVP & Performance Engineering Story</h4><div><br />When we are going to take MVP as early as possible to market, there is a tradeoff. What are considered in subsequent priorities which will be compromised on negotiation by engineering and business?<p></p><p>The context matters when prioritizing! Be it for performance or functionality or security or any quality criteria.</p><p>I will interpret the question asked from the point view of a MVP's deployment or publishing.<br /></p><ul><li>Are you asking why I have picked MVP?</li><ul><li>The performance is contextual and it is based out on multiple touchpoints, its boundaries and interfaces of a system.</li><li>I cannot talk on all those in this blog post.</li><li>Neither, I want to talk about the KPIs and metrics. </li></ul><li>I want to share which you can pick, consume, and apply in your work.</li></ul><p></p><p style="text-align: justify;"><br />Now, we have prioritized performance for a MVP. Aren't we? Prioritized means, it matters, it concerns us and we are okay to compromise on few for it. Let us jump to Left with MVP in our hands to identify the requirements of business.</p><p style="text-align: justify;">As a business, we will have a rough idea on how we are pitching and selling our services and to whom. As a test engineer, you can sense what is the key transaction [business work flow] in the MVP. You will know the touchpoints, interfaces and boundaries in the architecture that communicates and work together to keep MVP delivering the value. Don't you?</p><p style="text-align: justify;">Say the business wants the MVP to support and serve 500 requests per second. I should know about the <b>500 requests</b> here.</p><p style="text-align: justify;">Is it,<br /></p><ul><li>Concurrent Requests?</li><li>Active Concurrent Requests?</li></ul><br />This matters! Both are not the same. Have you asked this question? <b>It is a requirement we miss to capture</b>.<br /><div><br /><br /></div><div><br /></div><p></p><h3>Capturing Performance Questions for a MVP</h3><p style="text-align: justify;">It is about the awareness for first! How much am I opening up myself to the awareness? This brings an energy and it is contagious.</p><p style="text-align: justify;">How do I bring the performance awareness in my team so that it is engineered into the system we develop? This is a culture drive to an organization.</p><p style="text-align: justify;">Now, I know, the MVP has to serve 500 concurrent active users in a second per business's expectation to meet its reach and target. If I do not know this, I have to capture this data, for first. How do you capture?</p><h4><br /><br />A Use Case to Ponder</h4><p>One use case which would trigger the spark of thinking is - <i><b>How should Disney+ Hotstar's services perform to live stream the India vs Pakistan Cricket World Cup 2023 on 14th Oct 2023?</b></i></p><p></p><ul><li>How should it capture the performance and its requirement for this day?</li><ul><li>How should this system scale to crores of viewers streaming the live video of the match?</li><li>How should this system scale for the gamification - emoji, chat and other viewers engagement during the crores of viewers making requests from client interface?</li></ul></ul><div style="text-align: justify;">Try to play the past 30 minutes of this live video? What did you see? Why? That is part of the performance engineering strategy!</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">This use case opens up the different topics of Architecture and Performance Engineering. Be aware of it and explore on them. This is not what we want to talk, now. We want to talk on a MVP and how to capture its requirements for having better performance experience.</div><div><i><br /></i></div><div><br /></div><div><br /></div><h4>Step Up by 5% Heuristic</h4><div><br /></div><div>On having a test environment which is close to the production context and the test data that looks realistic, I get started.</div><div><br /></div><div>I framed this "Step Up by 5% Heuristic" after few months on starting the practice of Performance Engineering and Testing. I failed, and I learned. I'm learning.</div><div><br /></div><div>I know, the expectation is 500 active concurrent users per second.</div><div><br /></div><div>I will start to evaluate the integrated systems of the MVP for the 5 percent of 500.</div><div><ul><li>What is the 5% of 500?</li><ul><li>I will start with 25 concurrent active users requests for the MVP.</li><ul><li>I will observe the emotional experience of when using the MVP during this time.</li><li>I will monitor and record the KPIs, and other needed data.</li></ul><li>Does it fail to serve 5% of concurrent active users?</li><ul><li>If failed, I know what to do now. <b>It helped me</b>.</li><ul><li>This helps me to draw the requirements better and rationally for the existing system's architecture.</li></ul><li>If it succeeds, <b>it helped me partially</b> in knowing what actually I wanted to know.</li><ul><li>I will raise the active concurrent users to 10%</li><li>That is, increasing it by 5%</li><ul><li><b><i>I repeat these tests until the MVP architecture lets me know about the requirement it needs for the performance of serving 500 active concurrent users in a second</i></b></li><ul><li>Read the above sentence, again</li><li>The tests on MVP will let me learn what are its performance requirements for serving active concurrent 500 users in a given architecture, infrastructure, and tech stacks</li></ul></ul></ul></ul></ul></ul><div><br /></div></div><div><br /></div><p></p><h4>Beyond by 37% Heuristic</h4><div><br /></div><div style="text-align: justify;">I framed this "Beyond by 37% Heuristic" after I failed in framing the tests for performance. Talking the rationale of this heuristic is not the purpose this blog post. Let us catch up if you are curious and interested, we will discuss on this.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Do a salary hike of 30% indicate a high performance? I don't know! But 30% hike is something not commonly given to all, is what I see in my career so far.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">That said, this 37% has worked for me in the contexts I'm testing. Did it serve 685 (500 + 185) active concurrent users in a second? It helps me to draw a requirement analysis of the MVP system for this volume of concurrent active users.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Now, I will step up by another 37% of concurrent active users. That is, 870 (685 + 185) active concurrent users in a second.</div><div><ul><li>If seen, I have 1.5x traffic now. Did it serve?</li><ul><li>If yes, how many active concurrent users were served in a second? </li><li>I will correlate the KPIs of other integrated systems of a MVP.</li><ul><li> With the captured data and emotions</li><ul><li>This will tell, what should we expect despite what is the expectation from business</li><li>This difference will let us know "the requirements"</li><ul><li>How to gather information on -- what has to be optimized, changed, reorchestrated, eliminated, included, and more.</li><li>We start technically in establishing and framing the Performance SLAs and SLOs between the tech team and business.</li><li>Now the performance & its requirements will appear in the dots that are,</li><ul><li>Being connected</li><li>To be connected</li><li>To be disconnected</li><li>That does not exist</li></ul></ul></ul></ul></ul></ul><div><br /></div></div><div><br /></div><div>To conclude, shift wherever, take the performance engineering together! Revive its requirements to be healthier!</div><div><br /></div><div><br /></div><div><br /></div><div><u><b>Note</b></u>: You should read these blog posts if you have not:<br /><ol><li>Performance Testing: <a href="https://testinggarage.blogspot.com/2023/10/the-unspoken-kpi-and-missing-correlation-in-performance-engineering-and-testing-ravisuriya.html" target="_blank">Unspoken KPIs and The Missing Correlation</a></li><li>Architecture: The Common Shared Understanding -- <a href="https://testinggarage.blogspot.com/2023/10/what-is-software-architecture-how-it-enables-testability-and-testing-ravisuriya.html" target="_blank">Part 1</a></li><li>Architecture: Its Aid in Performance Engineering -- <a href="https://testinggarage.blogspot.com/2023/10/how-architecuture-helps-to-build-performance-engineering-and-testing-testability-ravisuriya.html" target="_blank">Part 2</a></li></ol><div><br /></div></div><div><br /></div></div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-7028399228927965110.post-14113846329777950462023-10-05T20:04:00.005+05:302024-02-05T10:51:09.061+05:30Architecture: The Common Shared Understanding -- Part 1<p style="text-align: justify;"> </p><p style="text-align: justify;">When we are developing a software system, <u>the requirement from a stakeholder is </u><b><u>not </u><i>'Fast'</i></b> or <b>'Scalable' </b>or <b>'Responsive'</b>. But, the stakeholder needs it and expects it. If you see, on a larger picture, the software system development and maintenance <i><u>is a job of balancing</u></i> too.</p><p style="text-align: justify;"><br />When a Software Architect [Technical Architect and Test Architect], works on architecting the software system and testing for the same,</p><p style="text-align: left;"></p><ul style="text-align: left;"><li style="text-align: justify;">It is about balancing the technical aspects with the business's requirements from stakeholders. </li><ul><li style="text-align: justify;">Do you see that?</li></ul></ul><p></p><p style="text-align: justify;"><br />Knowing the architecture of a software system and testing of same is one of a primary task for engineers on the project. Because, we software engineers have to balance it well. Balance, what? Balancing the technical aspects together with business's requirements from stakeholders.</p><p style="text-align: justify;"><br />This blog post is part of <u>100 Days of Skilled Testing</u> series. The second question posted for season 2 is,</p><blockquote><p style="text-align: left;"><i>How important is the understanding of application architecture to do performance testing better?</i></p></blockquote><p style="text-align: justify;"> <br /><br /></p><h3 style="text-align: justify;">What is an Architecture?</h3><p style="text-align: justify;">In context of Software Development & Engineering, the word "architecture" is one of the ambiguous words among the teams in a project and an organization. </p><p style="text-align: justify;">As a test engineer,</p><p style="text-align: justify;"></p><ul style="text-align: left;"><li>Did you ever had a discussion or arguments or debate with programmer and architect?</li><li>I had such discussions and I continue to have it today as well in the projects that I work.</li><ul><li>This is to know and understand</li><ul><li>What I should be doing as an engineer for first and as a Test Architect in the role?</li></ul></ul></ul><p></p><p style="text-align: justify;"><br />The outcome of this discussion showed me,</p><p style="text-align: justify;"></p><ul style="text-align: left;"><li>We all did not have a common understanding of it</li><ul><li>We did not share,</li><ul><li>"What I understood for the architecture and this architecture?"</li></ul></ul></ul><br />The primary goal of a Software System's Architecture is,<div><div><ul style="text-align: left;"><li>We all engineers on a project have a same understanding of it, in the aspects it exists for.</li><li>This understanding is arrived after we have put our thoughts into scrutiny and decided that we stick on to it, so that,</li><ul><li>We can balance well between the technical requirements and business expectation.</li></ul></ul>Are you with me, so far?<br /><div><br /></div><div><br /><br /></div><div style="text-align: left;"><b>A Software System's Architecture is</b>,</div><div style="text-align: left;"><ol style="text-align: left;"><li><u>A common shared understanding of what we all have for</u>,</li><ul><li> What we are developing, testing, and to about maintain?</li></ul><ol><ul><li>And, Why? Who? When? Where? How?</li></ul></ol><li><u>Represents the boundaries and interfaces of what matters</u>,</li><ul><li>That is [to be] orchestrated, designed, implemented, how it communicates, and what it will have, and not.</li><li>It also can show how the teams are structured and how the team and organization is organized.</li><ul><li>For example, in the Service Oriented Architecture, the teams are built and structured with respect to the service they offer.</li></ul><li>It is a model that is better than other models in a given context of technical requirements and business needs</li></ul><li><u>The context and awareness for,</u></li><ul><li>Why it is the way it is?</li><ul><li>The cost and value for being so.</li></ul><li>What to do when it has to be changed? Where to change?</li><ul><li>How simple and quick to change?</li><ul><li>What are the cost and value for being so?</li></ul></ul><li>How can I monitor and observe all these consistently?</li></ul><li><u>The Gateway of Testability - it tells what is the Testability available for,</u></li><ul><li>Letting know what is critical and priority to test</li><li>Where, How, When, and What tests can be framed, designed, and executed? To what extent?</li><ul><li>Why these tests?</li><li><b>If an architecture does not talk about and do not have the Testability, we have a serious problem! </b>This has to be fixed for first on priority.</li><ul><li>An architecture has to provide the Testability and Programmability scope and opportunities to develop a software system that is of value!</li></ul></ul></ul></ol><div><br />For today, this is my understanding for the "Architecture".</div><div><br /></div><div>I'm a Test Architect in the role and I expect myself to be an hands-on engineer for first. It is a necessity for an architect to be an hands-on engineer.</div><div><br /></div><div><br /></div><div><br /><br /><u>Note</u>: Read the <b>Part 2</b> of this blog post <b><a href="https://testinggarage.blogspot.com/2023/10/how-architecuture-helps-to-build-performance-engineering-and-testing-testability-ravisuriya.html" target="_blank">here</a></b>.</div><div><br /></div><div><br /></div></div><p></p></div></div>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-7028399228927965110.post-5872348039641471962023-10-05T20:04:00.004+05:302024-02-05T10:50:50.081+05:30Architecture: Its Aid in Performance Engineering -- Part 2<p style="text-align: justify;"> </p><p style="text-align: justify;">I hope, you and I have the common shared understanding for the word "architecture". If not read <a href="https://testinggarage.blogspot.com/2023/10/what-is-software-architecture-how-it-enables-testability-and-testing-ravisuriya.html" target="_blank"><b>this</b></a> blog post and come here to know about the dots.</p><p style="text-align: justify;"><br /></p><h3 style="text-align: justify;">Do I Know the Dots?</h3><p style="text-align: justify;">Before connecting the dots, I should know,</p><p></p><ul style="text-align: left;"><li style="text-align: justify;">What are the dots and how to identify them, where and when? </li><li style="text-align: justify;">Who can help me in doing so?</li></ul><div style="text-align: justify;"><br />In Software Security & Engineering, we use a Threat Model to,</div><div style="text-align: justify;"><ul><li>Identify the risks, surface area, tests and to develop the payloads. </li><li>A software system's architecture will help in developing and improvising the Threat Model consistently.</li></ul></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">I see, the same for software system's Performance Engineering & Testing. To test better for the performance,</div><div style="text-align: justify;"><ul><li>I need to identify the dots, risks, surface area, tests, payloads, monitoring aid and correlation of all these.</li><li>The understanding of software system's Architecture is a necessity to do so.</li><ul><li>But, what are the dots here?</li></ul></ul></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">The dots can be identified when I know how to use the Testability provided by the architecture. <u><i><b>This leads me to evaluate the performance for the boundaries and interfaces in isolation and as a whole, and then correlate</b></i></u>.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br />With this, it puts me to question - <b>What is the performance of this architecture?</b></div><div style="text-align: justify;"><ul><li>I did not say the performance of software system; I said, the architecture. </li><li>There should be some characteristics to identify and evaluate the performance engineering models offered by the architecture. </li><ul><li>What are they? </li></ul></ul></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">The architecture's characteristics helps,</div><div style="text-align: justify;"></div><div style="text-align: justify;"><ul><li>To identify and distinguish the dots in ease and to test better.</li><ul><li>How can I test for the performance aspects of a software system using the architecture's characteristics?</li><li>How do I identify these characteristics in the architecture of software system?</li></ul></ul></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br /></div><h3 style="text-align: justify;"><br />The Characteristics of an Architecture</h3><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Last year I read an article from ByteByteGo System Alliance. This article flashed in my mind as I read the question,</div><blockquote><div style="text-align: justify;"><i style="text-align: left;">How important is the understanding of application architecture to do performance testing better?</i></div></blockquote><div style="text-align: justify;">This is one of the <a href="https://medium.com/bytebytego-system-design-alliance/top-10-architecture-characteristics-non-functional-requirements-with-cheatsheat-7ad14bbb0a9b" target="_blank"><b>article</b></a> which I refer to identify the performance characteristics of the architecture. I refer to the cheatsheet shared in this article for my references.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">In few projects and organizations that I have worked, most of these characteristics were put into practice and production environment. I monitored them in usual traffic and unexpected traffic. The feel is something that I cannot describe in words; I want to experience it.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br /><h3>Performance Engineering Aided by Architecture</h3><div><br /></div></div><div style="text-align: justify;">Which characteristics of architecture is associated to the which boundaries and interfaces of a software system? Knowing this, helps you and me in thinking - <i><u>What has to be tested in performance for this interface in this boundary?</u></i></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">I want to share my work experience here. But, I see, if I share something which we all can relate to, it will be of help in knowing - <i>Why it is important to know about the architecture to do the performance testing?</i></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Below one is a recent use cases from software industry for the same.</div><div style="text-align: justify;"><ul><li>I will not explain in detail; but, I will bring the key points to the context of this blog post</li></ul></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><span style="text-align: left;"><br /></span><h4><span style="text-align: left;">Amazon Prime's Audio-Video Monitoring Service Moving to Monolith Architecture</span></h4><div style="text-align: justify;"><span style="text-align: left;">What I and You Should Know:</span></div><div><ul><li style="text-align: justify;"><span style="text-align: left;">The complete Amazon Prime system did not move to Monolith.</span></li><ul><li style="text-align: justify;"><span style="text-align: left;">The <u>Prime Video's Audio-Video Monitoring Service</u> moved to Monolith.</span></li><ul><li style="text-align: justify;">Why?</li><ul><li style="text-align: justify;">This monitoring system which was orchestrated with a Microservice architecture did not scale after a limit</li><li style="text-align: justify;"><b>Problem Statement</b>:</li><ul><li style="text-align: left;">Say, the Prime Audio Video Monitoring Service expected a load of active 100 concurrent users streaming movie Kantara in Kannada audio.</li><li style="text-align: left;">After the 6th user started streaming the same video [in the same audio or different audio language], this monitoring system did not scale to include other 95% of concurrent user. </li><li style="text-align: left;">As a result, the Prime Audio Video Monitoring Service slows down [or stops,] and eventually the video streaming to the active concurrent users will take a hit. </li><li style="text-align: left;">This monitoring service is important so that each user gets a video and audio of the agreed upon quality and streaming.</li></ul></ul></ul></ul></ul><div style="text-align: justify;">What I understand is,</div><div style="text-align: justify;"><ul><li>This monitoring service was continued in production while the team looked for better solution in performance with the given architecture.</li><li>While it did so, the cost of having this architecture was high when it had to scale up.</li><li>Looks like Prime Video business beared this cost for sometime is what I see.</li></ul>But, the online streaming business cannot settle and agree to pay high cost, while it is planning to stream the live sports action in coming days. A need came to look into performance characteristics in the being used monitoring service's architecture.</div></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">It eventually re-orchestrated the existing components with a new architecture in place.</div><div style="text-align: justify;"><ul><li>It moved from the distributed microservice system to a monolith system, where the spawning of Amazon Step Function (error detector clusters) happened vertically.</li><li>Along with this, the architecture of this monitoring service was placed in a way, such that, most of its components came into one process.</li><li>Thus, eliminating the S3 bucket as immediate storage for video frames (as images) and audio files.</li><li>This architecture helped the creational, behavioral, structural and functional characteristics of Prime Video's Audio-Video Monitoring service.</li></ul></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">Prime Video <a href="https://www.primevideotech.com/video-streaming/scaling-up-the-prime-video-audio-video-monitoring-service-and-reducing-costs-by-90" target="_blank">says</a> upon testing for performance and changing to monolith by rearranging the existing components,</div><div style="text-align: justify;"><ul><li>It saved 90% of the cost.</li><li>90% of cost for Amazon, is what in the numbers if it is in Indian Rupees? </li><li>How much a tester gets paid if just 1.5% of this 90% is paid as a salary per month?</li><ul><li>I will leave this to your thoughts and calculation.</li></ul></ul></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">If I know the architecture and where to look for what characteristics, it helps me to think of right performance tests for the context. </div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">The Hostar's emoji introduction in live cricket matches during 2018 and its consistent improvisation in processing for performance is a good use case, to the question -- <i>Why it is important to know about the architecture to do the performance testing?</i></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br /></div><div style="text-align: justify;">To conclude, architecture cannot be ignored in Testing. It plays a critical role for aiding and identifying the testability and BCFS (behavior, creational, functional and structural) characteristics of a software system.</div><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><br /></div></div><p></p>Unknownnoreply@blogger.com0