How do I interpret the Task Statistics in Website Testing?

In this help, we'll discuss how to use the Task Statistics analysis to:

  • View Task Statistics per task
  • Discover the Most Common Pages
  • Evaluate Time Taken
  • Interpret the Success score and gauge its statistical accuracy

View Task Statistics per task 

  • This analysis displays Task Statistics one task at a time
  • Use the task selection dropdown list in the top section of the screen to switch to the task you want to analyze.
  • Alternatively, use the arrow buttons to the left and right of the dropdown list to move to the previous/next task.

For each task, Task Statistics contain:

  • The task's text, starting URL and the correct destinations
  • Pie chart of task results

The pie chart and its legend display the types of results as such:

  • Success - the respondent ended the task on a correct URL
  • Failure - the respondent ended the task on an incorrect URL
  • Skip - the respondent pressed the button to skip the task
  • Cancel - the respondent closed the task tab

Discover the Most Common Pages 

These are the various types of pages most commonly visited by the respondents. Reading through them will help you better understand how users behaved while solving each task.

  • Most Common Success Pages - the pages on which respondents most commonly ended the task with success. Not shown in case when success is judged by an exact match to one specific URL, as in that case all successes have to be on that one URL.
  • Most Common Fail Pages - the most common pages where respondents have incorrectly judged the task to be done.
  • Most Common Abandon Pages - the pages where respondents most often abandoned the task.
  • Most Common First Clicks - which pages were most often opened with the first click - immediately after visiting the starting page for the first time (a.k.a. the most common second visited page while solving the task).

Evaluate Time Taken 

The bottom section of the Task Statistics contains the Time Taken and Success scores.

  • Time Taken is represented by a candlestick chart. The bar illustrates the upper and lower quartiles with the line inside marking the median value
  • T-shaped 'wicks' in the candle chart mark the highest and lowest times that it took to complete the task

Interpret the Success score and gauge its statistical accuracy 

  • Success is the percentage of answers that were correct, calculated for all answers from all respondents. It shows how well the respondents handled the tasks
  • Aside from the score itself, the charts also contain information about its statistical accuracy:

The upper and lower limit represent the bounds of the confidence interval - an interval which tells you how accurately the metric represents all of your respondents. The more narrow this interval is, the more we can be sure that the metrics of success or directness are statistically accurate. (The confidence interval is calculated through the adjusted Wald method with the confidence level of 95%. Adjusted Wald was chosen because it’s the best method to use when there are 150 result samples or less. For information, try out this calculator and read the text below)

Naturally, to achieve results with more statistical accuracy, you need to recruit more respondents to your study.