Task score
Every task receives a score out of 10. Aim to achieve an 8 or higher as sign of good performance. Lower scores point to the presence of user experience issues.
In this help, we’ll discuss how to use the Task Statistics analysis to:
- View Task Statistics per task
- Evaluate Time Taken
- Interpret Success and Directness scores and gauge their statistical accuracy
View Task Statistics per task 
- This analysis displays Task Statistics one task at a time
- Use the task selection dropdown list in the top section of the screen to switch to the task you want to analyze.
- Alternatively, use the arrow buttons to the left and right of the dropdown list to move to the previous/next task.
For each task, Task Statistics contain:
- The task’s text and the correct answers
- The task’s Overall score (calculated as a rounded 3-to-1 weighted average of the squares of success and directness scores) is colored based on how acceptable it’s considered to be: green (great score – 8 and higher), light green (good score with some caveats – 6, 7), orange (bad score – 4, 5) or red (terrible score – 3 and lower)
- Pie chart of task results
The pie chart and its legend display the types of results as such:
- Direct success – the respondent went directly for the correct answer
- Indirect success – the respondent took a different path and backtracked before going for the correct answer
- Direct failure – the respondent went directly for a wrong answer
- Indirect failure – the respondent took a different path and backtracked before going for a wrong answer
- Direct skip – the respondents skipped the task immediately, without clicking anywhere in the prototype first
- Indirect skip – the respondent explored the prototype before choosing to skip the task
Evaluate Time Taken
The bottom section of the Task Statistics contains the Time Taken, Success and Directness scores.
- Time Taken – The time from when the respondent sees the task until they submit their answer.
- Time Taken is represented by a candlestick chart. The bar illustrates the upper and lower quartiles with the line inside marking the median value
- T-shaped ‘wicks’ in the candle chart mark the highest and lowest times that it took to complete the task
Interpret Success and Directness scores and gauge their statistical accuracy
- Success is the percentage of answers that were correct, calculated for all answers from all respondents. It shows how well the respondents handled the tasks
- Directness is the percentage of answers that were direct (without backtracking), calculated for all answers from all respondents. It shows how sure the respondents were with their answers
- Aside from the scores themselves, the charts also contain information about statistical accuracy of the scores:
The upper and lower limit represent the bounds of the confidence interval – an interval which tells you how accurately the metric represents all of your respondents. The more narrow this interval is, the more we can be sure that the metrics of success or directness are statistically accurate. (The confidence interval is calculated through the adjusted Wald method with the confidence level of 95%. Adjusted Wald was chosen because it’s the best method to use when there are 150 result samples or less. For information, try out this calculator and read the text below)
Naturally, to achieve results with more statistical accuracy, you need to recruit more respondents to your study.
Time Taken, Success and Directness
How long did the task take respondents to complete this task. Higher completion times are usually a warning sign. The success rate informs you about how many people found a right destination and how many failed to do so. It doesn’t matter if they found the solution immediately or if they had to backtrack.The directness rate informs you about how many people reached their destinations directly, without taking a different path first, getting lost or backtracking. It doesn’t matter if they actually found a right answer or not.
