Counterbalancing is the bread and butter of experimental design. One issue that frequently comes up in online experiments is block randomization. Suppose there is a between-subjects condition that has four different levels, A, B, C, and D. In block randomization, the first four participants will be assigned (in random order) to groups (or levels) A, B, C, and D. Then, for the next four participants, we again repeat this procedure, and assign the block of four participants (in random order) to groups A, B, C, and D.

For example, the first 8 participants might be assigned as follows: A, D, C, B, B, A, D, C. In this example, there are exactly two participants per group and therefore the condition is balanced.

With this procedure, we ensure that as we add more participants, we always balance the number of participants per group as best as possible.

For online experiments, block randomization is trickier than one might realize. The problem is that when you post a study on Prolific or MTurk, a group of participants might immediately start the experiment. We can easily assign the first couple of participants who arrive, but a potential problem occurs when participants don't finish the experiment. A participant might simply give up and navigate away to a different website or they might leave their browser tab open. Even though these participants were originally assigned to a group, we need to remove those participants from our counts such that the final set of participant data will be balanced.

The FirebasePsych functions blockRandomization and finalizeBlockRandomization take care of this complexity for you.

Here is an example of how to get an assignment assignedCondition to a condition we named viewPoint. This condition has 4 different levels, and we assign each participant to one of the levels (numDraws=1). The FirebasePsych library function blockRandomization will start a table in the realtimedatabase that will keep track of the number of participants assigned so far to each level and drawing conditions that will balance the design. It also takes care of participants who started the study but didn't finish within a certain time required (60 minutes in this example).

// How many minutes before a participant will time out (i.e., when we expect them not to finish the study)?
// We need this (rough) estimate in order to zero out the counts for participants that started the study
// but did not finish within this time (and will therefore never finish)
const maxCompletionTimeMinutes = 60;

// Example 1: Assign a random condition for Viewpoint
const viewPointCondition = 'ViewPoint'; // a string we use to represent the condition name
let numConditions = 4; // Number of conditions for this variable
let numDraws = 1; // Number of  assignments (mutually exclusive) we want to sample for this participants
let assignedCondition = await blockRandomization(studyId, viewPointCondition, numConditions,
  maxCompletionTimeMinutes, numDraws); // the await keyword is mandatory

Importantly, we also need to make one more call to the FirebasePsych function finalizeBlockRandomization in order to finalize the counts if the participant actually reaches the end of experiment. This is needed to ensure that participants who go through the whole experiment will get their assignments to conditions counted while the participants who do not finish will have their assignments available for allocation to new participants.

Example:

if (endofStudy) {
  finalizeBlockRandomization(studyId, viewPointCondition);
}

Limitations

In the current implementation, if any participants were posthoc rejected from the study (e.g., manual inspection led to removal of the participant), their assignments to conditions are still preserved in the internal database and the design will not be fully balanced. It is non-trivial in the current implementation to remove these participants manually.

Full code example ("demoFirebasePsych_blockrandomization")

<!DOCTYPE html>
<html lang="en">
<head></head>
<body>
<div id="message"></div>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
<script src="./firebaseconfig.js"></script>
<script src="./firebasepsych1.0.js" type="module"></script>
<script type="module">
/// Importing functions and variables from the Firebase Psych library
import {
  writeRealtimeDatabase, writeURLParameters, readRealtimeDatabase,
  blockRandomization, finalizeBlockRandomization, firebaseUserId
} from "./firebasepsych1.0.js";

//-------------------------------------------------------------------------------------------------------
//   Demonstrate block randomization (between subjects)
//
//   functions: blockRandomization() and finalizeBlockRandomization()
//
//   Goal: Randomly assign participant to a condition with the constraint to balance total counts across conditions
//
//   In the example below, we are randomly assigning each participant conditions:
//   1) Viewpoint: four different levels, assign each participant to one of the levels
//   2) ObjectType: three different levels, assign each participant to one of the levels
//   3) imageChoice: twelve different levels, assign each participant to 4 of the 12 levels
//
//   Note that the blockrandomization will return conditions indices that are zero-indexed 
//------------------------------------------------------------------------------------------------------

// Declare which study we are running
const studyId = 'study1';
console.log("Firebase UserId=" + firebaseUserId);

// How many minutes before a participant will time out (i.e., when we expect them not to finish the study)?
// We need this (rough) estimate in order to zero out the counts for participants that started the study
// but did not finish within this time (and will therefore never finish)
const maxCompletionTimeMinutes = 60;

// Example 1: Assign a random condition for Viewpoint
const viewPointCondition = 'ViewPoint'; // a string we use to represent the condition name
let numConditions = 4; // Number of conditions for this variable
let numDraws = 1; // Number of  assignments (mutually exclusive) we want to sample for this participants
let assignedCondition = await blockRandomization(studyId, viewPointCondition, numConditions,
  maxCompletionTimeMinutes, numDraws); // the await keyword is mandatory
let msg = viewPointCondition + ": participant is assigned level: " + assignedCondition;
console.log(msg);
$('#message').append(msg + '<br>');

// Example 2: Assign a random condition for ObjectType
const objectTypeCondition = 'ObjectType'; // a string we use to represent the condition name
numConditions = 3; // Number of conditions for this variable
numDraws = 1; // Number of  assignments (mutually exclusive) we want to sample for this participants
assignedCondition = await blockRandomization(studyId, objectTypeCondition, numConditions,
  maxCompletionTimeMinutes, numDraws);
msg = objectTypeCondition + ": participant is assigned level: " + assignedCondition;
console.log(msg);
$('#message').append(msg + '<br>');

// Example 3: Sample 4 out of 12 images
const imageCondition = 'imageChoice'; // a string we use to represent the condition name
let numImages = 12; // Number of total images
numDraws = 4; // Number of images to sample for this participant
let assignedImages = await blockRandomization(studyId, imageCondition, numImages,
  maxCompletionTimeMinutes, numDraws); // the await keyword is mandatory
for (let i = 0; i < numDraws; i++) {
  let msg = "Image " + (i + 1) + " of " + numDraws + ": participant is assigned image index: " + assignedImages[i];
  console.log(msg);
  $('#message').append(msg + '<br>');
}


// Experiment will run here...
//

// End of experiment has been reached...
let endofStudy = true;

// Important: at the end of the Experiment, if the participant completes successfully, we have to make some final database updates.
// Why? Because if a participant completes the experiment, we treat that differently for the purpose of counterbalancing than a participant
// who aborts the experiment (e.g. times out or navigates to a different page)
// The following function makes sure that the state for this participant reflects the fact that the participant has completed the experiment.

if (endofStudy) {
  finalizeBlockRandomization(studyId, viewPointCondition);
  finalizeBlockRandomization(studyId, objectTypeCondition);
  finalizeBlockRandomization(studyId, imageCondition);
  $('#message').append('Finalizing block randomization tables...<br>');
}

</script>
</body>

</html>

Results when running the script

Every time you run this script (e.g. when you refresh the browser window), the browser and console window shows the random assignment that was made to two different between subject conditions.

Important: the numeric value of the condition is zero-based. For example, if there are four possible levels for a condition, the levels returned by the counterbalancing function range from 0 to 3. In other words, if the numbers represent letters for the conditions, we have A=0, B=1, C=2, and D=3.

Internal representation

If you are curious, below is an illustration of how these functions store information on the realtime database to keep track of the condition counts. Note that this information is stored under the child node studyInfo which is shared among all participants. Under object type, the child nodes refer to different participants (e.g. aeasst4bzkt00ayfmzis7e6a). These are random id's (not the same as the firebase user ids) used to keep track of which group was assigned to which participant and when they started the experiment.