Skip to main content

Metrics

Metrics allow you to understand your customers and how the service is performing. Measuring benefits/value before and after transformation will help you to articulate the overall success of your service re-design.

Qualitative vs quantitative metrics

It is important to understand that there is a difference between qualitative and quantative metrics.

  Qualitative Quantitative
Conceptual Concerned with understanding human behaviour from the customer’s perspective. Concerned with discovering facts about social phenomena. 
Methodological  Data is collected through interviews and participant observations. Data analysed by themes from descriptions by customers. Data is collected through measuring things. Data analysed through numerical comparisons and statistical inferences. 
Examples Gathers information that is not in numerical form. i.e. diary accounts, open-ended questionnaires, unstructured interviews and unstructured observations.  Gathers information in numerical form, e.g. number of transactions, cost per transaction, number of customers affected, cycle time, number of pages, etc. 

When to gather and understand metrics

Metrics are applied in all phases - initially to understand your customers, then as a measurement to realise benefits.

Phase Why Qualitative or Quantitative
Pre-mobilisation To qualify the problem statement and the level of value. Quantitative 
Mobilisation To document additional benefits identified during workshops. Quantitative
Discovery

Interviews and observations to understand customers. 

Quantify hypotheses to determine the best one to take forward.

Qualitative

Quantitative 

Alpha To ensure benefits are still being realised. To develop a benefits road map. Quantitative
Beta  To measure user satisfaction, digital take-up, completion rate and cost per transaction. Quantitative

How to collect metrics

Metrics can be collected in a number of ways:

  • Websites: adding Google anaytics can provide behavioural statistics like page views, informing which pages people are viewing, or not.
  • System: where forms or applications are entered into a system it can provide information on the number of applications received over a period, or cycle times, or anything else that can be collated from the information collected.
  • Paper-based forms: these can still provide information simply by counting the number over a period or picking key items from the information collected. This is a very laborious process.
  • Service centres: if your service is supported over the counter, you could record the number of visits, what services are being processed, how long it generally takes, what questions are being asked?
  • Post office: if your service can be processed via the post office, you could record the number of transactions per service, types of users that use post offices and costs involved. 
  • Emails: if your service involves an email component, these can be counted and can record why the email was sent in the first place?

Example metrics

Measurement  Description Example
 Transaction volumes  The number of transactions that are processed by a service over a period of time, e.g. quarterly, yearly.  20,000 per year
Cross counter The number of customers that physically present and have their service processed over the counter at a service centre over a period of time, e.g. quarterly, yearly. 50,000 per quarter
Cycle time How long does it take to complete a transaction? This is measured from the time an application is sent in for processing through to its completion. 14 days
Call centre How many calls are received, about what, and the average time per call. 15 per day, 7 minutes per call
Web analytics Can provide numerous statistics like page views, browsers, locations, exit pages. Page views, bounce rates, browser used, location, exit pages, etc.
Cost per transaction What is the current cost per transaction? This is measured by collating all costs involved in processing the transaction and dividing it by the number of transactions processed over that same duration.  $2 per transaction
 Social benefit  Benefit to people of the environment, such as fuel consumed for a typical person driving to and from a service centre, or kilometres travelled.

884,626 litres                                                                                                                       

7.2 million kms

Team skills required to capture metrics and data

A good understanding of how to measure a service is required. It's important to ensure your measurements are correct as they could determine which problem you will or will not move into Alpha. This task is usually left to anyone who has a good understanding of the service, already does statisitical measurements or is a business analyst.

Last updated: 23 September 2016