Gamified Learning Can Be Effective

It should not be surprising that the Department of Defense (DoD) is exploring the potential of “gamified learning” to enhance education in fields such as foreign language study. What might be surprising is that gamification can make learning something as dry, but important, as defense acquisition fun and effective. While more research is needed, recent studies at the Naval Postgraduate School (NPS) suggest both are possible.

What Is Gamification?

Conventional wargames and simulations improve strategic and tactical thinking, flight training, and weapon skills training, among other things. Applying gamification to acquisition, however, is a novel innovation.

Gamification occurs when the means of learning are infused and enhanced with game-like elements: fantasy or simulated environments; competition; points, leaderboards, and badges; and other features common to competition. Done right, gamified methods enhance learning through increased motivation and engagement, which can result in deeper processing and better retention than conventional modalities.¹

High-risk, tightly regulated, zero-defect environments (such as the military) offer little room for experimentation and put a high price on failure that can stifle learning. This presents a paradox: How do organizations promote effective and deep learning in professional fields in which the conditions most supportive of learning are perceived as a risk to the ultimate mission?

The answer is to decouple the operational environment from the learning environment.

“Serious/simulation” games put the player through the motions of performing real-world tasks in a simulated but realistic environment. These games closely recreate physical and relational environments, as in the case of sophisticated flight simulators or the popular Apex Officer police-training game.

In “exposure” games, players practice the skills and abilities of their real-world roles through proxy or carry-over effects. For example, players in survival games such as 7 Days to Die must rely on resource management and planning over time to survive, even though the post-apocalyptic game environment bears little resemblance to the real world. And not all gaming is purely digital. For example, professional military instructors have advocated for exposure to gaming using traditional board games to teach team-building skills.²

Finally, “engagement” games introduce relevant subject matter to players in an alternate universe/setting to evoke increased interest and engagement.

Balancing common game features promotes greater learning by reducing aspects of the operating environment detrimental to learning, reinforcing those that promote critical learning objectives within guardrails. For example, fantasy aspects can encourage students to risk failure while including elements of real-world limitations, such as legal constraints. And they can counterbalance exploration with critical thinking, representation, and rules that represent highly regulated environments. In short, games with challenges/goals, curiosity/mystery, and feedback succeed when they are entered into voluntarily, offer “do-overs,” and allow for freedom of exploration and decision.

Shoot to Skill

In our work, students at NPS created a first-person shooter game involving gun battles and bomb defusing, in which success depended on players’ ability to correctly answer questions about federal acquisition rules and regulations. Our learning partners at the 344th Training Squadron at Lackland Air Force Base, Texas, provided MBA students with an approved acquisition curriculum and assisted in developing pre- and post-tests for assessing short-term knowledge retention among the training squadron’s newest enlisted students.

At the start, an on-screen manager introduces relevant information needed to answer curriculum questions correctly. Following the instruction period, a player learns that the office is under siege. Entering the main game area, players must fight off waves of attackers in each level. At critical junctures, players must defuse a bomb by cutting one of four wires, corresponding to four possible answers to a federal acquisition–related question. If the single correct wire is cut, the bomb is defused and the player earns points to put toward upgraded equipment. If the player answers incorrectly, the bomb explodes and the player takes damage. At the end of the game each player receives a report detailing their performance on attacker engagement and bomb defusing—i.e., correct answers and time.

Learning for gaming participants was compared to outcomes for control groups who received the same material in standard format—a guided PowerPoint lecture. Along with reviewing post-training knowledge tests, we compared learners’ satisfaction and engagement to understand the opportunities and challenges of gamified learning in military education.

What We Learned

Groups that received gamified training showed mixed results in short-term material retention compared with the control groups. In particular, variation existed among different gamified study waves.

In three of the five waves, the control group (traditional method) outperformed the game-playing group by a median improvement of 5 to 10 percent. In the fourth wave, the gamified and traditional methods performed equally well. In the fifth, which varied environmental and curriculum elements, gamified methods outperformed traditional ones by more than 15 percent.

Three factors may explain the fifth wave’s improvement.

• Students come to education and training with a range of preferences for learning modalities. Some prefer almost any form of game over traditional instruction, while others prefer only specific types of games. Still others find gamified learning entirely undesirable. This suggests gamified methods might not be a replacement for all traditional methods. The fifth wave consisted of students electing to participate.

• Another important source of variation was the gamified and conventional learning environments. Four waves of research conducted at the 344th used Chromebooks not gaming computers, which provide better graphics and smoother performance. Chromebook-exposed student comments indicate this negatively affected their experiences.

In the fifth wave, at NPS, players used gaming computers. This wave had the best outcomes. Students reported enjoying the experience and had fewer complaints. This suggests that using appropriate technology to support gamified learning is critical. In addition, students at the 344th played individually observed by instructors, whereas NPS players were allowed to interact and engage in competitive practices, with the MBA study team standing by for technical assistance. Taken together, these environmental variations probably meaningfully affected student performance.

• A final source of variation may have been in the retention assessment. For the four waves at the 344th, pre-, post-, and in-game questions matched exactly. All versions randomized the sequence of questions such that players could not simply memorize a pattern of answers to beat the game.

For the fifth wave, at NPS, answers within the game did not match exactly what were on the pre- and post-tests; answers could be derived from questions within the game, requiring higher-level, critical thought, not rote repetition. The fifth wave treatment group also had zero detractors based on a customer experience tool measures from Net Promoter^©.

The Final Boss

Gamified learning is ripe for future development, research, and investment. Our team at NPS is working to build new games related to acquisition sciences. One avenue for future research is how different types of games (e.g., first-person shooter and role-playing games) can succeed by adhering more closely to individual player preferences. We anticipate a future in which a range of curricula are offered to students through a suite of options—an app store.

Gamified instruction and education is not ideal for all students, any more than lectures are for visual learners or reading assignments are for auditory learners. Voluntary participation is a key tenet for success. Indeed, mandatory play may limit the benefits of gamified learning.³ Our findings suggest that gamified learning as an augmentation tool may be the best approach for most situations and curricula. Our randomized assignment of students to gamified vs. traditional methods may have negatively impacted the performance of those not predisposed to their assigned group. Future studies should consider allowing for self-selection despite this being contrary to most design recommendations for a “between groups” study. Understanding gamer types is critical for designing games that will sell, or—in our case—be adopted.

Which brings us to one of the greatest challenges with using games for education and training. Various cells in and outside DoD are working on games to promote learning in military education—NPS, Air University, North Carolina State University, Defense Acquisition University, Defense Language Institute, and a small band of organic developers within the Air Force Installation Contract Center; each is working independently with limited cross-talk. The opportunity for collaboration is at our fingertips. The time is ripe for researchers across the defense enterprise to “level up,” making the potential of game-based learning a reality for military education.