Task score
Every task receives a score out of 10. Aim to achieve an 8 or higher as a sign of good performance. Lower scores point to the presence of user experience issues.
This section covers how to use the Task Statistics analysis to:
- View Task Statistics per task
- Evaluate Time Taken
- Interpret Success and Directness scores and gauge their statistical accuracy
View Task Statistics per task 
- This analysis displays Task Statistics one task at a time.
- Use the task selection dropdown list in the top section of the screen to switch to the task you want to analyze.
- Alternatively, use the arrow buttons to the left and right of the dropdown list to move to the previous/next task.
For each task, Task Statistics contain:
- The task’s text and the correct answers.
- The task’s Overall score (calculated as a rounded 3-to-1 weighted average of the squares of success and directness scores) is colored based on how acceptable it’s considered to be: green (great score – 8 and higher), light green (good score with some caveats – 6, 7), orange (bad score – 4, 5) or red (terrible score – 3 and lower).
- Pie chart of task results.
The pie chart and its legend display the types of results as such:
- Direct success – the respondent went directly for the correct answer.
- Indirect success – the respondent took a different path and backtracked before going for the correct answer.
- Direct failure – the respondent went directly for a wrong answer.
- Indirect failure – the respondent took a different path and backtracked before going for a wrong answer.
- Direct skip – the respondents skipped the task immediately without clicking anywhere in the tree first.
- Indirect skip – the respondent explored the tree before choosing to skip the task.
Evaluate Time Taken
The bottom section of the Task Statistics contains the Time Taken, Success, and Directness scores.
- Time Taken The time from when the respondent sees the task until they submit their answer. If you selected Prompt respondents to start each task with a button, this time is calculated from the press of the button.
- Time Taken is represented by a candlestick chart. The bar illustrates the upper and lower quartiles with the line inside marking the median value.
- The T-shaped ‘wicks’ in the candle chart mark the highest and lowest times that it took to complete the task.
Interpret Success and Directness scores and gauge their statistical accuracy
- Success is the percentage of answers that were correct, calculated for all answers from all respondents. It shows how well the respondents handled the tasks.
- Directness is the percentage of answers that were direct (without backtracking), calculated for all answers from all respondents. It shows how sure the respondents were with their answers.
- Aside from the scores themselves, the charts also contain information about the statistical accuracy of the scores.
The upper and lower limits represent the bounds of the confidence interval – an interval that tells you how accurately the metric represents all of your respondents. The more narrow this interval is, the more we can be sure that the metrics of success or directness are statistically accurate. (The confidence interval is calculated through the adjusted Wald method with a confidence level of 95%. Adjusted Wald was chosen because it’s the best method to use when there are 150 result samples or fewer. For information, try out this calculator and read the text below)
Naturally, to achieve results with more statistical accuracy, you need to recruit more respondents to your study.
Time Taken and Success
How long the task took respondents to complete this task. Higher completion times are usually a warning sign. The success rate informs you about how many people found the right destination and how many failed to do so. It doesn’t matter if they found the solution immediately or if they had to backtrack.
Directness
The directness rate informs you about how many people reached their destinations directly without taking a different path first, getting lost, or backtracking. It doesn’t matter if they actually found the right answer or not.
