Testing

EVALUATION MATRIX

45 minutes

A big sheet of paper, minimum A3, or a whiteboard of a flip chart
Paper, pencils or markers of different colours
Sticky notes
Evaluation data, research data and service specifications

What is this tool and what is its purpose and benefit?

The evaluation matrix is a tool for prototype evaluation and decision-making based on testing results. It is a table that lists the prototypes or prototype versions to be evaluated and the evaluation criteria that are important in delivering the final service, such as the user-orientation, usability, foreseen costs, viability and feasibility of the prototype, and this way helps visualising which of the prototypes has most of the required criteria. The matrix helps to compare the quality and appropriateness of the prototypes, and see which required elements should still be covered, or which ones should be improved before the final development of the service.

Steps how to use this tool in practice

The same steps apply when working individually, in pairs or in a group.

STEP 1

Take a big sheet of paper, a whiteboard or a flipchart and add on the top of it the name of your project. For example, the name of the service under development, and draw two axis and a matrix on the paper or a flipchart.

STEP 2

Add the prototypes or prototype versions on the other axis, for example the vertical one, naming and numbering them, each on their own column.

STEP 3

Define the service/design criteria that are important in delivering the final service and add these on the other axis, each on their own row. These come from earlier defined research data and service specifications. The criteria should be the same as the testing criteria, but also new ones may be added if relevant issues come up during testing. You can also add a score (preferable on scale 1−5) to each criteria depending on their importance.

STEP 4

Take the testing data for each prototype and start adding it onto the matrix each criterion data on the respective row and under the respective prototype. You can write this data on paper/whiteboard or on sticky notes.

STEP 5

Once ready, look at the matrix and see which protype fills in most, and the most important criteria (scoring can help in this). You can also highlight the important elements with different colour markers. Also, spot all the elements that are weak or still missing. You can use a different colour for these.

STEP 6

Make the final prototype selection based on the results of your matrix, which prototype seems to be of the most suitable quality for the service and for final development. You can also combine elements of other prototypes into the selected one, or create a new one with the best features of all the prototypes.

Tips and hints for using this tool

There might not always be enough prototype versions for the same service.
This tool can also be used for selecting concepts and ideas or deciding on the research and evaluation questions.