An Experiment in Group Study

Group study is very important to us at Grockit. In fact, it’s something we’re staking our name and our business on: the notion that studying together is better than studying alone. While we’re not alone in believing that group study is important and have heard plenty of anecdotal stories about how people like studying with others, we needed some solid experimental evidence. We believe that it’s critical to test and validate our assumptions, especially assumptions about the core learning experiences in Grockit. We wanted to see whether group study actually impacted people’s studying: did they stay on task longer, study more, or study more effectively? Looking at how people study on Grockit, the answer to all of these seems to be yes. However, because we let learners direct their own studying on Grockit, we can’t separate the effect of studying in a group from a person’s choosing to study in a group, which might show biases according to the learner’s ability or intent. So when we couldn’t find any studies which looked at the sort of group study we’re providing, we decided to create our own.

The experiment is fairly straightforward. We randomly selected a portion of newly arriving Grockit learners and put them into the testing group, giving them a slightly modified Grockit experience. Instead of letting people select whether to study in a group or alone, we gave them a single button: “Study Now”. Each time they chose to study, we’d look and see whether there were other people studying the same thing. If there were, we’d treat this as an experimental case that we could test and flip a coin to see what study condition this would go in. Half the time, we put the learner into a solo game, studying on their own; the other half, we put the learner into a group game, studying with other people.

comparison of minutes spent in game, for group vs. solo study
We found three key results supporting the hypothesis that group study is more engaging and effective for learners:

  • We found that learners studied longer when they were in a group study session than when they were alone. Studies have shown that time on task is a key predictor of study effectiveness. It’s a simple correlation: studying longer is more effective. By providing a more engaging study experience, learners have more fun and spend longer studying. On average, learners spent 3x as long studying when they were in a group study session as when they played alone. The boxplot is even more striking: you can see that learners spend more time studying in group games across the board. (For my fellow stats geeks out there, the Mann-Whitney-Wilcoxon Rank-sum test results in Z = 8.3883, with a p-value < 2.2e-16. That's pretty irrefutable, as statistics go.)
  • We found that learners studied over 2x as many questions when they were in a group study session as when they were alone. This was important because we wanted to make sure that studying with others wasn’t just making people slower to answer questions. But learners study more than twice as many questions, on average, when in group study than when they study alone.
  • We found that learners were more likely to answer questions correctly when they were in a group study session than when they were alone. It could have been that learners were simply goofing off in group study, and that they were socializing rather than seriously paying attention to the material. But in fact, the opposite was the case with Grockit learners, where the social pressure is to focus on the material. Learners in group games actually answered correctly more often than when they were playing solo.

We want to find the most effective ways to help people learn, and that requires making sure the results are valid and verifiable. We’re working on a peer-reviewed paper to provide full results and data from our experiment; if you want access in the meantime, please write to Grockit was founded on the belief that we could come up with a better way to help people study, and that a key part of doing that was helping learners teach and learn from each other. We’re glad to have some strong evidence to support this belief, and will continue to test our theories to build an even better system.

Try 50 questions now FREE
First Name