Advanced randomization of answer options

What is advanced randomization in surveys?

Advanced randomization in surveys is a technique by which the answer options are presented to each respondent in a different order. It is used to overcome order bias - a behavior in which respondents tend to select the first option. To tackle this, survey creators present answer options such that they need to spend some time going through all the options and choose an honest answer.

Prevent order bias using advanced randomization

Often respondents want to complete surveys as soon as possible, leading them to look for the first answer that makes sense. If the first choice is the relevant answer, they may choose it and move to the next question while ignoring the remaining options. Presenting answer items in the same sequence for each respondent can generate a bias towards those options at the top. Hence, in the survey report, the weight of these answers gets overstated.

Example of advanced randomization in surveys

Consider a coffeehouse that wants to conduct a customer feedback survey and asks respondents to select their favorite coffee. The survey creator can randomly display different kinds of coffee, such that respondents do not view them in the same order. It will ensure that the respondents get different coffee as the first option.

Uses of advanced survey randomization

• Concept testing: Use advanced choice randomization to test new concepts before investing resources in them. Respondents can be shown various options in random order and asked to select one.
• Feedback surveys: Another application of option randomization can be feedback surveys in which survey participants can be shown their commonly used products or services as the answer options. Changing the sequence for all respondents will ensure that respondents don't over-select a particular option.
• Online tests: Academic institutions can randomize answer items in online tests and quizzes to force students to go through all the options.

Types of randomization in surveys

Order bias can creep in at various levels while answering the survey. Survey creators can overcome this challenge by configuring randomization in a survey software at below levels.

• Block: A survey can be organized into a group of questions known as a block. You can use a block randomizer to display a set of questions in no fixed order. Further, you can also present questions on the page randomly within a block.

Question: A randomizer question lets you display survey questions randomly to the respondents. You can configure question randomization at two levels and alter question order.

Randomly display one question

Randomize the sequence of a list of questions

• Answer options: Display answer items randomly to the survey respondents.

We offer three options to randomize answer options.

Simple: Randomize the order of all options. It is classified into three categories.

Default: Answer items are displayed in the same sequence as set by the survey creator.

Ascending: Answer options are sorted in the ascending order alphabetically.

Descending: Answer options are sorted in the descending order alphabetically.

• Random: Randomize the order of few options out of all the options.
• Advanced randomization: Create subgroups of answer choices and display the groups randomly.

Advanced randomization options

• Current display order: Answer items will be displayed in the same sequence as presented in the list.
• Randomize all choices: Options will be inserted randomly into the question in the places labeled {random}.
• Random sub-selection: Answer items will be inserted randomly into the question in the places labeled {random subset}. You can also specify the number of options to be inserted.
• Do not display: Hide the answer options that should not be presented to the respondents.

After saving the settings, the answers options will be randomized based on the rules configured. You can preview the randomizing answer order with the survey preview URL.

Note: After a respondent completes the survey, the answer items will randomize for the next respondent. However, if the same respondent refreshes the page or goes back in the same browser, the answer items will not randomize.

How to preserve randomization in surveys?

There can be research surveys with multiple questions having the same set of answer items. In such cases, you need to preserve the random order used to display the options in the first question so that the same sequence of options is used in the following questions.This phenomenon is known as extraction.

Learn more: How to set up extraction logic in a survey question?

How to download randomized answer options in surveys?

You can generate reports and analytics on the surveys with random answer options. You can also download survey data in common formats like .xls, .pdf, and .doc.

You can view which options were shown to the respondents and what were their chosen answers. However, you will not be able to view the exact sequence in which the choices were presented to the respondents.

Advantages of advanced randomization

• Prevent order bias: Research studies have shown that respondents show bias to the first option they are presented. So, displaying answer options in no fixed order can help to get a more accurate idea of how they feel.
• Prevent skewed data: If any option is overly selected by most respondents, it can skew the overall survey results. Randomizing answer options ensure the accuracy of the data collected.
• Better quality of data: Gain insights that can be trusted. Advanced randomization lets survey creators collect high-quality data.
• Avoid survey fatigue: Going through the options of the same type can lead to survey fatigue. Presenting answers in no fixed order helps maintain the interest of the respondents.

How to set up advanced randomization in your surveys?

Learn how to set up and use this feature with our help file on advanced randomization of survey answers in surveys.