How do I interpret the Questionnaire Analysis in Website Testing?

In this help, we'll discuss how to use the Questionnaire analysis to:

  • View questions and answers
  • Group free-text answers
  • Hide unused answers

View questions and answers 

  • The Questionnaire analysis contains the results for the Screening question, Pre-Study and Post-Study questionnaires and the After Task questionnaires. Select which questionnaire you want to view in the tabs in the top part of the screen.
  • In the case of After task questionnaire, use the task selection dropdown list in the top section of the screen to switch to the task you want to analyze.
  • Alternatively, use the arrow buttons to the left and right of the dropdown list to move to the previous/next task.
  • By default, all questions in the questionnaire are displayed at once. You can choose to display one question at a time by selecting it in the question selection dropdown list.
  • You can also move between questions by pressing the arrow buttons on either side of the question selection dropdown list.
  • Each question displays its order, type, text and answers, along with the number of respondents who gave that answer (total and percentage)

Net Promoter Score 

  • If a question is a Net Promoter Score question, besides a bar chart of scores, you'll also find additional stats:
  • Answers are split into three groups based on the score. Each group provides a number of respondents and a percentage:
    • Promoters - Score 9-10 - Users who are happy with using your website/service/etc. and spread the word to others.
    • Passives - Score 7-8 - Users who are content but have their reserves. Fix what ails them to decrease the chance of their defecting to competition.
    • Detractors - Score 6 and lower - Unhappy users who may spread negative word of mouth. Pay close attention to what they have to say.
  • Net Promoter Score - The final score is calculated as the percentage of promoters minus the percentage of detractors. Its value is a number between -100 (if all respondents are detractors) and 100 (if all respondents are promoters). The higher the value, the better.

Group free-text answers 

  • With free-text questions, you can turn on Group similar answers to consider similar answers as the same answer (this resolves issues with uppercase/lowercase and typos automatically, e.g. 'bank', 'Bank' and 'bamnk' get grouped together)
  • Set the Degree of similarity to adjust how similar two answers have to be in order to be considered as the same answer
  • The higher the Degree of similarity, the stricter the grouping will be (the Degree of similarity is used as a bottom threshold on Dice's Coefficient within the group)
  • Under the ellipsis button, turn off Set Degree of similarity value as global to set the Degree of similarity for each question individually

Hide unused answers 

Under the ellipsis button, turn on Hide answers with no responses to hide those answers in multiple-option questions that no respondents picked from the analysis.

Skip Logic 

If you have Skip Logic enabled in your questionnaire you will be able to see the selected outcome for each available option in the results. The possible outcomes include:

  • Skipping to a question - this can be any other question in the questionnaire. It will be denoted using a link, which takes you to the results of that specific question after clicking on it.
  • QUESTIONNAIRE END - selecting this option will immediately end the questionnaire for the participant. This can mean the end of the study as a whole or moving on to the tasks in the main body of the study.