Complexity Decision Framework (CDF) & it's importance in Test Suite Prioritisation and…

Complexity Decision Framework (CDF) & it's importance in Test Suite Prioritisation and Categorisation

Is Test Case 'Priority' & 'Complexity' is same or different? If different, is there any relation between them? How do we measure the 'Complexity' of a Test Case? Can we leverage both to create the perfect test suite?

After searching for many years, I barely found any standard framework which could answer these questions. To solve all these problems, I have come up with 'Complexity Decision Framework (CDF)', which would help us calculate Test Case complexity using some specific attributes and how this could be used as a 'Secondary' driver for Test Suite Prioritisation and Categorisation.

Complexity Decision Measurement Attributes:-

For simplicity, currently, I have considered only two (2) attributes to measure the 'Test Case Complexity': -

A) Human Resource Input to execute test case (X) [Measurement Unit — Time] — This is the overall approximate time needed to execute the test case from the tester's perspective only or could be the mean/average value of multiple testers. In the below table, I have identified a few relevant conditions to each attribute and assigned an appropriate number for each condition, where the magnitude of 'Assigned value' increases if it increases the overall/total unit value.

E.g., A TC with five steps can take 10 mins to execute, whereas a TC with 15 steps can take 25 mins. Even vice-versa is also possible. Considering the overall execution time, relevant parameter value needs to be considered.

B) Reciprocal Dependency b/w Multiple Stakeholder/Supplier (Y) [Measurement Unit — Time] — This is the information exchange time between multiple stakeholders & suppliers, Turnaround ETA, Action enablement etc. Similar to the above attribute, the table below will give some ideas on condition identification and relevant value mapping.

E.g., could be some product & price configuration in the back-office, data flow dependency to downstream components etc.

Please note: Both Human Resource Input (X) & Reciprocal Dependency (Y) are mutually independent variables, and changing one parameter doesn't impact the other.

Complexity Decision Value (CDV):-

In this step, we calculate the Complexity Decision Value (CDV), which is the addition of values of Human Resource Input (X) and Reciprocal Dependency (Y) mentioned above. So, CDV = (X+Y), which is a numeric value for any test case.

E.g. — (X + Y) = (1+1)=2 or (2+3)=5 or (3+4)=7 etc. depending on above two tables. This value will be used in the next section to categorise different complexity types.

Complexity Decision Matrix (CDM):-

The primary objective is to develop a matrix in table format where the different ranges of CDV will be used for complexity segmentation. As the CDV ranges can start from Min. 2 to Max 8, I have segregated it into 4 Core categories below.

Also, to simplify, I have placed the value of both the attributes (X & Y) like below to prepare a matrix & the colour code signifies a specific type of Complexity with the CDV.

Test Suite Categorisation Quadrants (2x2 Matrix):-

In this step, the objective is to come up with a 'Test suite categorisation quadrant' using a 2x2 matrix and to help the selection considering 'Complexity' along with 'Priority'. This will lead us to identify each quadrant with a specific category and prioritise the overall suite for maximum testing coverage. The primary goal is to create a 'CLOSE to PERFECT' Test and Regression suite to achieve maximum TEST COVERAGE of scenarios in correct PRIORITY order. This will help us to track the percentage (%) of important/critical test cases coverage for a limited execution duration/period.

As mentioned above, the 'Priority' of any test case is nothing but the 'Priority of the Requirements. It is Business and Product Manager/IT Business Analyst Driven in most cases. Any HIGH Priority requirements or User Story makes the relevant test case priority to HIGH by default.

Please note: We are NOT replacing 'Priority' with 'Complexity'; instead, integrating another driver using Categorisation Quadrants will be more meaningful and effective. The priority remains the 'Primary Driver', and Complexity is the 'Secondary Driver'.

Category-1: -

This combination mixes' HIGH Priority' and 'Low & Medium Complexity' Test cases. This category of Test cases will be considered to execute in the first phase.

Reason: — 'Low & Medium Complex' TCs will take less time to execute comparing 'High & Critical Complex' TCs and will provide more test coverage and assurance than any other categories.

Category-2: -

This combination mixes' HIGH Priority" and "High & Critical Complexity" Test cases. This category of Test cases will be considered to execute in the second phase.

Reason: — 'High & Critical Complex' TCs will take a higher time to execute compared to Category-1 TCs and will provide less test coverage & assurance compared to Category-1.

Category-3: -

This combination mixes of 'Low Priority' and 'Low & Medium Complexity' Test cases. This category of Test cases will be considered to execute in the third phase.

Reason: — 'Low Priority' TCs will have lesser importance comparing 'High Priority' TCs and take lower time to execute comparing Category-2 TCs, but it is less critical comparing Category-1 & Category-2 as they deal with only HIGH priority requirements.

Category-4: -

This combination mixes of 'LOW Priority' and 'High & Critical Complexity' Test cases. This category of Test cases will be considered to execute in the last phase at any time.

Reason: — 'Low Priority' TCs will have lesser importance comparing 'High Priority' TCs (Category-1 & 2) and take higher time to execute comparing 'Low Complex' TCs (Category-1 & 3). Also, 'High & Critical Complex' TCs will take a higher time to execute compared to Category-1 TCs and will provide less value to the product considering testing coverage & assurance comparing any other categories.

Advantages & Benefits of this framework:-

1. We are reducing waste from the testing life cycle, which will, in turn, increase the testing efficiency.
2. Test estimation can be improved by the project team drastically due to the simple mathematical robust calculations.
3. It can leverage when the timelines are tight and similar confidence can be achieved with lesser test execution.
4. In the case of repetitive testing, some of the 'higher complexity & low Priority' may not be run. And this can lead to saving some effort in turn to Cost optimisation.
5. Lesser time in test execution using 'Categorisation Quadrants'.

So, in summary, I feel this framework will help us prioritise scenarios to ensure critical/important functionalities are tested early in the testing life cycle, which will automatically lead to the early detection of defects. And the most important is the faster time to market to satisfy customer needs with proper MVP and CVP.

Important note:- This framework is completely customizable as per project needs.

Keep watch on further knowledge-sharing posts on product management from me :-)

Feel free to connect with me on LinkedIn.

Complexity Decision Framework (CDF) & it's importance in Test Suite Prioritisation and… was originally published in Bootcamp on Medium, where people are continuing the conversation by highlighting and responding to this story.